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PREFACE 


This  report  was  prepared  by  the  Institute  for  Defense  Analyses  (IDA)  for  the  Office 
of  Engineering  Technology,  Deputy  Under  Secretary  of  Defense  (Research  and  Advanced 
Technology)  and  the  Air  Force  Human  Resources  Laboratory,  Logistics  and  Human 
Resources  Division,  under  Contract  number  MDA  903  84  C  0031,  Task  Order  T-D6-553, 
"Applications  of  Systems  Engineering  Techniques  to  Development  of  a  Unified  Life  Cycle 
Engineering  Environment."  The  issuance  of  this  report  meets  the  specific  task  of  surveying 
"techniques  which  have  been  used  in  past  studies  of  design  processes." 

This  report  was  reviewed  by  Drs.  Jeffrey  H.  Grotte  and  Frederick  R.  Riddell  of  the 
Strategy,  Forces  and  Resources  Division  of  IDA. 
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EXECUTIVE  SUMMARY 


The  aim  of  this  study  was  to  gain  a  deeper  understanding  of  engineering  design 
through  the  evaluation  of  the  methods  used  to  study  the  design  process.  The  study  focused 
on  determining  the  methods  needed  to  represent  and  analyze  the  design  process.  This 
report  identifies  and  evaluates  methods  used  by  the  design  research  community  that  could 
be  used  to  establish  requirements  for  improving  the  effectiveness  of  the  engineering  design 
process. 

A.  BACKGROUND 

Unified  Life  Cycle  Engineering  (ULCE)  is  a  design  engineering  environment  in 
which  the  quality  of  a  product  is  improved  by  integrating  consideration  of  producibility  and 
supportability  with  design  attributes  for  performance,  cost,  and  schedule.  The  need  for 
ways  to  model  and  analyze  the  design  process  in  an  ULCE  environment  became  apparent 
during  the  ULCE  Architecture  and  Integration  Requirements  Study  conducted  at  the 
Institute  for  Defense  Analyses  (IDA)  during  fiscal  year  1987  and  funded  by  the  Air  Force 
Human  Resources  Laboratory  [Ref.  1].  This  study  consisted  of  examining  and  developing 
alternative  architectures  embodying  ULCE  concepts.  From  the  beginning  of  the  study,  it 
was  clear  that  the  engineering  design  process  is  extremely  complex,  even  for  relatively 
simple  products.  The  architecture  developed  during  the  study  is  proposed  as  a  conceptual 
framework  for  controlling  this  complexity.  This  architecture  called  for  the  application  of 
meta-design,  the  design  of  the  design  process  or,  more  precisely,  the  design  of  the  design 
decisionmaking  process. 

The  meta-design  concept  developed  from  the  realization  that  the  design  process  is 
not  generic,  except  at  a  very  high  level.  The  design  process  must  be  flexible,  since  the 
design  process,  and  in  particular  the  design  decisionmaking  process,  is  driven  by  the 
requirements  of  the  product.  There  will  always  be  more  than  one  way  to  design  a  product 
and  more  than  one  sequence  of  decisions  to  choose  from.  The  alternate  design  processes 
must  be  evaluated  to  determine  the  best  method,  and  to  properly  evaluate  the  alternate 
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design  processes,  some  representation  of  the  design  process  or  the  design  decisionmaking 
process  is  required.  Research  into  the  methods  used  to  model  the  design  process  was  seen 
as  the  next  essential  step,  and  this  research  was  funded  by  the  Air  Force  Human  Resources 
Laboratory,  Logistics  and  Human  Factors  Division,  during  fiscal  year  1988.  The  inherent 
assumptions  of  this  research  are  that 

•  ULCE  environments  must  be  based  on  formal  models  of  the  design  process. 

•  A  rigorous  understanding  of  design  is  necessary  to  develop  formal  models  of 
the  design  process. 

Additional  reasons  why  studying  the  design  process  is  of  such  importance  to  LILCE 
have  been  identified  throughout  the  authors'  research  activities  at  IDA.  If  any  portion  of 
the  design  process  is  to  be  automated  or  if  design  tools  are  to  be  efficiently  integrated  into 
the  design  process,  then  the  design  process  itself  must  be  fully  understood.  Additionally, 
the  ULCE  objectives  will  be  accomplished  only  through  adequate  design  education. 
Design  cannot  be  taught  or  learned  without  a  thorough  understanding  of  how  designers 
t.iink  and  function  in  the  design  environment.  Many  of  the  methods  detailed  in  this  report 
address  these  issues  as  well. 

B.  DESIGN  RESEARCH  STUDIES 

A  number  of  methods,  used  in  past  research  studies,  describe  the  design  process. 
These  methods  may  be  used  to  study  different  levels  of  design  activity.  Three  levels  of 
design  activity  have  been  distinguished  [Ref.  2]  as  in  Table  ES-1. 

In  this  report,  design  activity  at  levels  1  and  2  is  considered  to  represent  individual 
design  activity,  and  design  activity  at  level  3  is  considered  to  represent  organizational 
design  activity.  Although  these  two  types  of  activity  are  distinct,  they  are  interdependent. 
Both  are  essential  to  understanding  the  design  decision  process. 

When  studying  design  as  an  individual  activity,  one  is  interested  in  how  an 
individual  designer  uses  innate  problem-solving  capabilities  to  solve  the  design  problem, 
independent  of  the  design  environment.  These  studies  attempt  to  understand  the  cognitive 
decision  situations  faced  by  a  designer  and  the  manner  in  which  personal,  empirical 
knowledge  (derived  from  both  formal  training  and  experience)  is  used  to  make  decisions 
and  solve  the  design  problem. 
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Table  ES-1.  Design  Activity 


Level 

Type 

Description 

1 

Decision 

Design  is  described  as  a  value-based  process  whereby  decisions 
depend  on  subjective  evaluations  made  by  individual  designers. 
Human  characteristics  of  individual  designers  determine  how  the 
design  activity  proceeds.  Decisions  made  at  this  level  concern 
minute  factors  in  the  design  task. 

2 

Product 

Design  activity  is  described  in  terms  of  design  methodologies. 
Descriptions  at  this  level  attempt  to  identify  and  characterize 
fundamental  processes  of  design  activity.  These  descriptions 
highlight  the  multit-de  of  decisions  that  must  be  made  to  create  a 
product. 

3 

Project 

Design  activity  is  described  in  terms  of  the  communal  activity  of  the 
team  or  organization. 

Organizational  level  design  studies  address  the  context  in  which  engineering  design 
IS  done  by  depicting  the  decision  situations  faced  by  all  elements  of  the  design 
environment.  The  design  organization  is  composed  of  a  number  of  diverse  functional  units 
(such  as  design,  management,  and  marketing)  and  is  affected  by  a  number  of  organizationa’ 
factors  (such  as  "the  nature  of  the  design  problem;  the  constraints  of  time,  resources,  work 
environment,  and  organization;  and  the  availability  and  support  effectiveness  of  design 
aides  and  tools."  [Ref.  3|)  Each  of  these  elements  influences  affects  the  product  life  cycle 
differently.  Organizational  level  design  studies  attempt  to  capture  the  nature  of  these  effects 
and  the  interaction  among  the  diverse  influences.  These  studies  develop  frameworks  for 
understanding  how  organizations,  on  a  macroscale,  make  design  decisions;  how  personal 
knowledge  is  affected  by  organizational  factors;  and  how  knowledge  is  shared 
(communication  achieved)  throughout  the  design  en'/ironment. 


More  than  50  research  projects  were  reviewed  to  develop  an  understanding  of  the 
scope  of  the  design  research  studies  throughout  the  1970s  and  1980s.  The  design 
disciplines  represented  by  this  collection  of  studies  are  diverse  and  include  mechanical, 
electronics,  civil  engineering,  architectural,  interior  ,  and  software  design.  While  most  of 
the  studies  were  led  by  academic  researchers,  a  few  were  performed  by  private  industry 
under  contract  to  the  Department  of  Defense. 


From  this  eclectic  assortment  of  research  projects,  18  were  selected  to  illustrate  the 
manner  in  which  the  design  decision  process  has  been  studied  by  the  design  research 
community  (Figure  ES- 1 ).  In  these  studies,  attempts  have  been  made  to  understand  design 
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Rssaarchar/ 

Institution 

Focus 

Date 

Published 

Bucciarelli/MIT 

Model  the  design  process  to  explain  the  role  of  tradibon,  habit, 
arxf  values  in  technical  discourse  and  decisionmaking 

1984 

Dyer  and  Flowers 

UCLA 

Formalize  rules  of  creabvity  in  the  area  of  mechanical  design 

1984 

Eckersley 

University  of  MD 

Analyze  the  verbal  behavior  of  designers  involved  in  problem¬ 
solving 

1988 

Rrxiler 

SUNY  Buffalo 

Analyze  the  role  of  analogical  reasoning  in  design  problem¬ 
solving  by  computers 

1981 

Fulton  and  Salley 
NASA 

Define  requirements  for  integrated  programs  for  aerospace  design 

1985 

Green  and  Brown 
Worcester  Polytech 

Analyze  the  krwwledge  used  and  problem-solving  behavior 
exhibited  during  roubrte  design  task 

1987 

Kalay.  Swerdloff.  and 
Harfmann 

SUNY  Buffalo 

Develop  fundamental  paradigms  of  design  as  a  compabble 
process 

1987 

Kurumatani  and 
Yoshikawa/University 
of  Tokyo 

Model  knowledge  of  design  obfects  and  processes  using  a  formal 
axiomatic  theory  of  design 

1987 

Love/GE 

Develop  a  framework  for  describing  variables  that  influence  the 
software  development  process 

1976 

Maher/Camegie 

Mellon  University 

Develop  a  domaln-irxfependent.  knowledge-based  approach  for 
preliminary  design  of  complex  systems 

1988 

Malhotra.  Thomas. 
Carroll,  and  Miller 

IBM  TJW  Research 
Center 

Identify  and  characterize  important  design  processes  to  gain  an 
understanding  of  aids  that  can  assist  the  design  process 

1978 

McCall/University 
of  Colorado 

Develop  a  design  method  tor  describing  design  processes  and 
attempting  to  improve  them 

1987 

Peterson,  Hagel. 
Nadler,  and  Chignell 

use 

Identity  process  and  decision  aids  in  design  through  an  analysis 
of  structured  interviews 

1988 

Petroski/Ouke 

Characterize  rational  design  through  the  analysis  of  historical  case 
studies  o'  failures 

1987 

Promisel,  Harfel, 
Kaplan.  Marcus,  and 
WhittenburgAJS  Army 
Research  Institute 

Assess  the  extent  to  which  human  factor  issues  were  considered 
throughout  the  development  process  of  four  weapon  systems 

1985 

Uliman,  DIetterich. 
and  Stauffer 

Oregon  State  Univ 

Develop  a  model  of  the  mecharxeal  engineering  design  process 
from  empirical  data 

1988 

Waldron  and  Waldron 
Ohio  State 

Conduct  a  retrosoeebve  study  of  the  mechanical  design  process 
to  id  ’tifv  decisions  made  during  conceptual  design 

1987 

Wallace/Cambridge 

■  'he  design  process  observed  in  context 

1987 

Figure  ES-1.  Oesiqr  Research  Studies  Considered  in  this  Report 


problem-solving  behavior,  identify  and  characterize  knowledge  used  to  make  design 
decisions,  or  represent  decision  structures  of  engineering  design  processes. 

The  methods  used  to  conduct  the  research  listed  in  Figure  ES-1  can  be  organized 
into  six  categories.  Each  category  represents  a  unique  approach  for  studying  the  design 
decision  process.  These  method:  include: 

•  Systems  analysis 

•  Field  observation  and  participation 

•  Retrospective  analysis 

•  Knowledge  representation 

•  Protocol  analysis 

•  Computational  experiments  of  problem-solving  behavior. 

The  first  three  methods  listed  provide  systematic  approaches  for  studying  the  design 
process  at  the  organizational  level.  The  last  two  methods  are  used  to  study  the  design 
process  at  the  individual  level.  The  knowledge  representation  method  can  be  used  at  both 
the  individual  and  organizational  levels;  in  this  report,  it  is  considered  at  only  the 
organizational  level.  Figure  ES-2  depicts  the  research  projects  listed  in  Figure  ES- 1  in  the 
context  of  the  classification  scheme. 

Various  methods  for  studying  design  at  the  organizational  level  are  described  in 
Chapter  II.  Chapter  III  describes,  in  detail,  some  methods  used  to  study  design  at  the 
individual  level.  Representative  studies  employing  the  method  are  given,  and  the 
advantages  and  disadvantages  of  using  each  method  are  outlined. 

C.  OBSERVATIONS 

1  .  Diverse  Motivations 

A  number  of  diverse  motivations  drive  the  study  of  the  design  process  from  both 
theoretical  and  practical  points  of  view.  For  example,  groups  of  investigators  throughout 
the  world  seek  to 

•  Establish  a  science  of  design 

•  Examine  underlying  mental  processes  and  cognitive  strategies  followed  during 
a  problem-solving  activity 
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Figure  ES-2.  Design  Research  Studies  in  Classification  Scheme 
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•  Understand  the  industrial  and  economic  implications  of  engineering  design 
processes 

•  Develop  techniques  that  could  help  designers  in  certain  stages  of  their  work, 
such  as  brainstorming,  synectics,^  value  analysis,  morphology,  and  lateral 
thinking; 

•  Develop  "systematic  procedures  to  guide  the  design  process"  (such  as  the 
German  VDI-Richtlinien)  [Ref.  4] 

•  Capture  design  experience  by  means  of  expert  systems,  etc.  [Ref.  4] 

•  Develop  improved  methods  for  teaching  design. 

The  abstracts  of  these  surveys  are  included  in  the  annotated  bibliography.  Anyone 
interested  in  engineering  design  research  should  examine  these  papers  in  detail  before 
initiating  related  work. 

2 .  Common  Themes 

Many  common  themes  underlie  the  research  reported.  These  commonalities  include 

•  The  engineering  design  problem  is  an  "ill-structured"  problem,"  and  thus 
difficult  to  solve  by  any  set  algorithm."  [Ref.  5] 

•  Design  problems  are  hierarchical  and  can  be  decomposed  into  subproblems. 

•  Design  is  both  a  creative  and  analytical  process. 

•  Design  is  a  dynamic  and  iterative  process. 

•  Design  problem-solving  behavior  involves  analogical  reasoning. 

•  "The  nature  of  designing  requires  an  interdisciplinary  approach."  [Ref.  6] 

•  Design  is  both  a  technical  and  behavioral  process;  it  cannot  be  studied  as 
though  it  were  one  and  not  the  other.  [Ref.  7] 

•  Although  a  number  of  attempts  have  been  made  to  define  engineering  design, 
none  has  been  completely  successful.  [Ref.  8] 

•  Designers  do  not  attempt  to  find  optimum  solutions,  rather  they  attempt  to  find 
solutions  that  work  ("satisfice"). 

•  "Many  researchers  opt  for  a  systems  approach."  [Ref.  9] 


*  Synectics  is  the  study  of  creative  processes,  especially  as  applied  to  the  solution  of  problems  by  a 
^  group  of  diverse  individuals.  Synectics  comes  from  the  Greek  word  meaning  the  joining  together  of 

different  and  apparently  irrelevant  elements. 
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•  Design  decisions  are  usually  not  independent. 

•  There  is  no  interdisciplinary  "language  of  design."  [Ref.  6] 

•  There  are  few,  if  any,  automated  aids  for  conceptual  design.  Most  design  aids 
are  applicable  during  detail  design. 

3 .  Common  Definitions 

Most  of  the  definitions  of  the  engineering  design  process  suggested  by  individual 
researchers  closely  match  those  of  the  Accreditation  Board  of  Engineering  and  Technology 
(ABET).  Engineering  design  is  described  by  ABET  as 

the  process  of  devising  a  system,  component,  or  process  to  meet  desired 
needs.  It  is  a  decision-making  process  (often  iterative),  in  which  the  basic 
sciences,  mathematics,  and  engineering  sciences  are  applied  to  convert 
resources  optimally  to  meet  a  stated  objective.  Among  the  fundamental 
elements  of  the  design  process  are  the  establishment  of  objectives  and 
criteria,  synthesis,  analysis,  construction,  testing,  and  evaluation.  [Ref.  8] 

4 .  Summary 

This  report  reviews  six  approaches  that  have  been  used  by  the  engineering  design 
research  community  to  examine  the  design  decision  process.  These  approaches  yield 
valuable  information  about  design  processes.  The  approaches,  however,  result  in 
significantly  different  types  of  design  decision  process  information.  Table  ES-2  is  a 
summary  of  the  types  of  design  decision  process  information  that  are  typically  produced. 

The  research  results  are  all  considered  models  of  the  design  decision  process. 
Although  the  natures  of  the  models  listed  in  Table  ES-2  are  quite  distinct,  each 
representation  serves  as  a  tool  for  considering  the  design  process  problem.  Because  none 
of  the  models  considers  all  aspects  of  the  engineering  design  decision  process,  a 
combination  of  approaches  is  required  to  study  design  processes  in  their  entirety.  As  this 
paper  illustrates,  the  engineering  design  research  community  is  exploring  the  application  of 
different  types  of  methodologies  to  advance  its  understanding  of  the  design  decision 
process.  Motivation  for  this  work  is  well  stated  by  Simon: 

There  are  no  more  promising  or  important  targets  for  basic  scientific 
research  than  understanding  how  human  minds,  with  and  without  the  help 
of  computers,  solve  problems  and  make  decisions  effectively,  and 
improving  our  problem-solving  and  decision-making  capabilities.  [Ref.  10] 
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Table  ES-2.  Design  Decision  Process  Information 


Approach 

Design  Decision  Process  Model 

Systems  Analysis 

Hierarchical,  systematic  diagrams  of  the  activities,  information  flows,  and 
influences  characterizing  a  typical  engineering  design  process 

Participant  Observation 

Quantitative  (graphs,  etc.)  and  qualitative  (narratives,  etc.)  descriptions  of 
the  activities  of  a  particular  design  project 

Retrospective  Analysis 

High-level  frameworks  describing  the  major  drivers  and  decision  points  of  past 
engineering  projects 

Knowledge 

Representation 

Formal  representations  of  design  decision  structures 

Protocol  Analysis 

High-level  frameworks  that  describe  the  basic  components  of  cognitive 
activity  associated  with  design  problem  solving 

Computational 

Experiments 

Executable  models  of  problem-solving  behavior 

I.  INTRODUCTION 


The  aim  of  this  study  was  to  gain  a  deeper  understanding  of  engineering  design 
through  evaluation  of  the  methods  used  to  study  the  design  process.  The  study  focused  on 
determining  the  methods  needed  to  represent  and  analyze  the  design  process.  This  report 
identifies  and  evaluates  methods  used  by  the  design  research  community  that  could  be  used 
to  establish  requirements  for  improving  the  effectiveness  of  the  engineering  design  process. 

A  survey  of  the  design  research  literature  was  conducted  to  determine  the  methods 
necessary  for  representing  and  analyzing  the  design  process.  From  this  survey,  various 
methods  used  to  model  the  design  process  were  identified  and  a  classification  scheme  of  the 
methods  was  developed.  After  examining  the  methods,  by  considering  the  results  reported 
in  the  literature,  the  strengths  and  weaknesses  of  each  approach  were  established. 

This  chapter  provides  background  information  (the  motivation  for  the  study)  and  an 
introduction  to  the  design  research  projects.  A  summary  chapter  of  observations  is 
provided  in  the  concluding  chapter. 

A.  BACKGROUND 

Unified  Life  Cycle  Engineering  (ULCE)  is  a  design  engineering  environment  in 
which  the  quality  of  a  product  is  improved  by  integrating  consideration  of  producibility  and 
supportability  with  design  attributes  for  performance,  cost,  and  schedule.  The  need  for 
ways  to  model  and  analyze  the  design  process  in  an  ULCE  environment  became  apparent 
during  the  ULCE  Architecture  and  Integration  Requirements  Study  conducted  at  the 
Institute  for  Defense  Analyses  (IDA)  during  fiscal  year  1987  and  funded  by  the  Air  Force 
Human  Resources  Laboratory  [Ref.  1].  This  study  consisted  of  examining  and  developing 
alternative  architectures  embodying  ULCE  concepts.  From  the  beginning  of  the  study,  it 
was  clear  that  the  engineering  design  process  is  extremely  complex,  even  for  relatively 
simple  products.  The  architecture  developed  during  the  study  is  proposed  as  a  conceptual 
framework  for  controlling  this  complexity.  This  architecture  called  for  the  application  of 
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meta-design,  the  design  of  the  design  process  or,  more  precisely,  the  design  of  the  design 
decisionmaking  process. 

The  meta-design  concept  developed  from  the  realization  that  the  design  process  is 
not  generic,  except  at  a  very  high  level.  The  design  process  must  be  flexible,  since  the 
design  process,  and  in  particular  the  design  decisionmaking  process,  is  driven  by  the 
requirements  of  the  product.  There  will  always  be  more  than  one  way  to  design  a  product 
and  more  than  one  sequence  of  decisions  to  choose  from.  The  alternative  design  processes 
must  be  evaluated  to  determine  the  best  method,  and  to  properly  evaluate  the  alternative 
design  processes,  some  representation  of  the  design  process  or  design  decisionmaking 
process  is  required.  Research  into  the  methods  used  to  model  the  design  process  was  seen 
as  the  next  essential  step.  The  inherent  assumptions  of  this  research  are  that 

•  ULCE  environments  must  be  based  on  formal  models  of  the  design  process. 

•  A  rigorous  understanding  of  design  is  necessary  to  develop  formal  models  of 
the  design  process. 

This  report  and  the  accompanying  report.  Managing  Engineering  Design 
Information  [Ref.  11],  provide  information  required  to  evaluate  those  models  for 
applicability  to  ULCE.  The  research  for  both  reports  was  funded  by  the  Air  Force  Human 
Resources  Laboratory,  Logistics  and  Human  Factors  Division,  during  fiscal  year  1988. 

Additional  reasons  why  studying  the  design  process  is  of  such  importance  to  ULCE 
have  been  identified  throughout  the  authors'  research  activities  at  IDA.  If  any  portion  of 
the  design  process  is  to  be  automated  or  if  design  tools  are  to  be  efficiently  integrated  into 
the  design  process,  then  the  design  process  itself  must  be  fully  understood.  Additionally, 
the  ULCE  objectives  will  be  accomplished  only  through  adequate  design  education. 
Design  cannot  be  taught  or  learned  without  a  thorough  understanding  of  how  designers 
think  and  function  in  the  design  environment.  Many  of  the  methods  detailed  in  this  report 
address  these  issues  as  well. 

B.  DESIGN  RESEARCH  STUDIES 

A  number  of  methods,  used  in  past  research  studies,  describe  the  design  process. 
These  methods  may  be  used  to  study  different  levels  of  design  activity.  Three  levels  of 
design  activity  have  been  distinguished  [Ref.  2]  as  shown  in  Table  I-l. 
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Table  1-1.  Design  Activity 


Level 

Type 

Description 

1 

Decision 

Design  is  described  as  a  value-based  process  whereby  decisions 
depend  on  subjective  evaluations  made  by  individual  designers. 

Human  characteristics  of  individual  designers  determine  how  the 
design  activity  proceeds.  Decisions  made  at  this  level  concern 
minute  factors  in  the  design  task. 

2 

Product 

Design  activity  is  described  in  terms  of  design  methodologies. 
Descriptions  at  this  level  attempt  to  identify  and  characterize 
fundamental  processes  of  design  activity.  These  descriptions  high¬ 
light  the  multitude  of  decisions  that  must  be  made  to  create  a  product. 

3 

Project 

Design  activity  is  described  in  terms  of  the  communal  activity  of  the 
team  or  organization. 

In  this  report,  design  activity  at  levels  1  and  2  is  considered  to  represent  individual 
design  activity,  and  design  activity  at  level  3  is  considered  to  represent  organizational 
design  activity.  Although  these  two  types  of  activity  are  distinct,  they  are  interdependent. 
Both  are  essential  to  understanding  the  design  decision  process. 

When  studying  design  as  an  individual  activity,  one  is  interested  in  how  an 
individual  designer  uses  innate  problem-solving  capabilities  to  solve  the  design  problem, 
independently  of  the  design  environment.  These  studies  attempt  to  understand  the 
cognitive  decision  situations  faced  by  a  designer  and  the  manner  in  which  personal, 
empirical  knowledge  (derived  from  both  formal  training  and  experience)  is  used  to  make 
decisions  and  solve  the  design  problem. 

Organizational  level  design  studies  address  the  context  in  which  engineering  design 
is  done  by  depicting  the  decision  situations  faced  by  all  elements  of  the  design 
environment.  The  design  organization  is  composed  of  a  number  of  diverse  functional  units 
(such  as  design,  management,  and  marketing)  and  is  affected  by  a  number  of  organizational 
factors  (such  as  "the  nature  of  the  design  problem;  the  constraints  of  time,  resources,  work 
environment,  and  organization;  and  the  availability  and  support  effectiveness  of  design 
aides  and  tools."  [Ref.  3])  Each  of  these  elements  influences  affects  the  product  life  cycle 
differently.  Organizational  level  design  studies  attempt  to  capture  the  nature  of  these  effects 
and  the  interaction  among  the  diverse  influences.  These  studies  develop  frameworks  for 
understanding  how  organizations,  on  a  macroscale,  make  design  decisions;  how  personal 
knowledge  is  affected  by  organizational  factors;  and  how  knowledge  is  shared 
(communication  achieved)  throughout  the  design  environment. 
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The  IDA  study  team  reviewed  more  than  50  research  projects  to  develop  an 
understanding  of  the  scope  of  the  design  research  studies  throughout  the  1970s  and  1980s. 
The  design  disciplines  represented  by  this  collection  of  studies  are  diverse  and  include 
mechanical,  electronics,  architectural,  interior,  and  software  design  and  civil  engineering. 
While  most  of  the  studies  were  led  by  academic  researchers,  a  few  were  performed  by 
private  industry  under  contract  to  the  Department  of  Defense. 

From  this  eclectic  assortment  of  research  projects,  18  were  selected  to  illustrate  the 
manner  in  which  the  design  decision  process  has  been  studied  by  the  design  research 
community  (Figure  I-l).  In  these  studies,  attempts  have  been  made  to  understand  design 
problem-solving  behavior,  identify  and  characterize  knowledge  used  to  make  design 
decisions,  or  represent  decision  structures  of  engineering  design  processes. 

The  methods  used  to  conduct  the  research  listed  in  Figure  I-l  can  be  organized  into 
six  categories.  Each  category  represents  a  unique  approach  for  studying  the  design 
decision  process.  These  methods  include 

•  Systems  analysis 

•  Field  observation  and  participation 

•  Retrospective  analysis 

•  Knowledge  representation 

•  Protocol  analysis 

•  Computational  experiments  of  problem-solving  behavior. 

The  first  three  methods  listed  provide  systematic  approaches  for  studying  the  design 
process  at  the  organizational  level.  The  last  two  methods  are  used  to  study  the  design 
process  at  the  individual  level.  The  knowledge  representation  method  can  be  used  at  both 
the  individual  and  organizational  levels;  in  this  report,  it  is  considered  at  only  the 
organizational  level.  This  classification  scheme  is  depicted  in  Figure  1-2.  Figure  1-3 
depicts  the  research  projects  listed  in  Figure  I-l  in  the  context  of  the  classification  scheme. 


Researcher/ 

Institution 

Focus 

Date 

Published 

Bucciarelli/MIT 

Model  the  design  process  to  explain  the  role  of  tradition,  habit, 
and  values  in  technical  discourse  and  decisionmaking 

1984 

Dyer  and  Rowers 

UCLA 

Formalize  rules  of  creativity  in  the  area  of  mechanical  design 

1984 

Eckersley 

University  of  MD 

Analyze  the  verbal  behavior  of  designers  involved  In  problem¬ 
solving 

1988 

Findier 

SUNY  Buffalo 

Analyze  the  role  of  analogical  reasoning  in  design  problem¬ 
solving  by  computers 

1981 

Fulton  and  Salley 
NASA 

Define  requirements  for  integrated  programs  for  aerospace  design 

1985 

Green  and  Brown 
Worcester  Polytech 

Analyze  the  knowledge  used  and  problem-solving  behavior 
exhibited  during  routine  design  task 

1987 

Kalay,  Swerdloff,  and 
Haufmann 

SUNY  Buffalo 

Develop  fundamental  paradigms  of  design  as  a  compatible 
process 

1987 

Kurumatani  and 
Yoshikawa/University 
of  Tokyo 

Model  knowledge  of  design  objects  and  processes  using  a  formal 
axiomatic  theory  of  design 

1987 

Love/GE 

Develop  a  framework  for  describing  variables  that  Influence  the 
software  development  process 

1976 

Maher/Camegie 

Mellon  University 

Develop  a  domain-independent,  knowledge-based  approach  for 
preliminary  design  of  complex  systems 

1988 

Malhotra,  Thomas, 
Carroll,  and  Miller 

IBM  TJW  Research 
Center 

Identify  and  characterize  important  design  processes  to  gain  an 
understanding  of  aids  that  can  assist  the  design  process 

1978 

McCall/University 
of  Colorado 

Develop  a  design  method  for  describing  design  processes  and 
attempting  to  improve  them 

1987 

Peterson,  Hagel, 
Nadler,  and  Chignell 

use 

Identify  process  and  decision  aids  in  design  through  an  analysis 
of  structured  interviews 

1988 

Petroski/Duke 

Characterize  rational  design  through  the  analysis  of  historical  case 
studies  of  failures 

1987 

Promisel.  Hartel, 
Kaplan.  Marcus,  and 
Whittenbur^'US  Army 
Research  Institute 

Assess  the  extent  to  which  human  factor  Issues  were  considered 
throughout  the  development  process  of  four  weapon  systems 

1985 

Ullman,  Oietterich, 
and  Stauffer 

Oregon  State  Univ. 

Develop  a  model  of  the  mechanical  engineering  design  process 
from  empirical  data 

1988 

Waldron  and  Waldron 
Ohio  State 

Conduct  a  retrospective  study  of  the  mechanical  design  process 
to  Identify  decisions  made  during  conceptual  design 

1987 

Wallace/Cambridge 

Model  the  design  process  observed  In  context 

1987 

Figure  1-1.  Design  Research  Studies  Considered  in  this  Report 
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Figure  1-2.  Classification  Scheme  for  Design  Research  Studies 
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Figure  1-3.  Design  Research  Studies  in  Classification  Scheme 
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II.  ORGANIZATIONAL  DESIGN  RESEARCH  STUDIES 


This  chapter  addresses  four  methods  for  investigating  design  decision  processes  at 
the  level  of  organizational  activity— systems  analysis,  field  observation,  retrospective 
analysis,  and  knowledge  representation. 

A.  SYSTEMS  ANALYSIS 

Systems  analysis  is  the  application  of  methodical  strategies  to  identify  and  describe 
both  significant  elements  of  a  complex  system  (or  process)  and  relationships  among  those 
elements.  To  apply  systems  analysis  techniques,  the  subject  under  examination  must  be 
amenable  to  decomposition  (i.e.,  it  must  comprise  discrete  elements  that  are  arranged 
hierarchically  and  can  be  independently  characterized). 

Since  the  inception  of  the  systems  movement,  both  formal  and  informal  systems 
analysis  strategies  have  been  used  to  build  multi-disciplinary  architectures  of  complex 
systems  and  processes.  One  classical  systems  analysis  architecture  views  complex  systems 
in  terms  of  inputs,  processes  (or  functions),  and  outputs.  Relationships  among  functions 
are  established  by  common  inputs  and  outputs.  This  is  a  useful  representation  for 
documenting  information  requirements  and  flows.  Variations  to  this  approach  include  the 
identification  of  enabling  mechanisms,  technologies,  constraints,  or  requirements.  Many 
formal  techniques  that  embody  this  architecture  have  evolved  over  the  last  decade.  These 
include 

•  Yourdon's  Structured  Analysis  (SA) 

•  Gane  &  Sarson's  Structured  Systems  Analysis 

•  Softech's  Structured  Analysis  and  Design  Technique  (SADT) 

•  Integrated  Computer-Aided  Manufacturing  (ICAM)  Definition  Languages 
(EDEFo,  IDEFi,  and  IDEF2) 

•  Nijssen's  Information  Analysis  Method  (NIAM) 

•  Michael  Jackson  Diagrams 


Wamier-Orr  Dic^grams 

Hierarchical  Input  Process  Output  (HIPO)  charts 
Nassi  Schneiderman  Charts. 


These  techniques  were  intended  to  expedite  requirements  definition  for  software 
development,  data  base  modeling,  process  design  and  control,  testing,  and  other  systems 
engineering  functions. 

Typically,  formal  systems  analysis  techniques  share  the  following  characteristics; 

•  The  process  is  viewed  from  a  top-down  perspective. 

•  The  system  is  decomposed  recursively  (high  level  elements  are  defined  in 
terms  of  constituent  subelements,  which  are  in  turn  defined  by  subelements, 
etc.). 

•  Elements  of  the  system  are  described  in  terms  of  a  hierarchical  input-process- 
output  architecture. 

•  The  system  under  study  is  represented  in  terms  of  a  graphical  notation 
language  with  predefined  semantics.  The  graphics  describe  hierarchies  of 
functions  and  relationships  among  those  functions. 

•  Some  methodologies  include  suggested  practices  for  data  collection,  data 
assimilation,  model  reviews,  and  revisions. 

Requirements  for  the  effective  application  of  systems  analysis  methods  to  modeling 
engineering  design  processes  include 

•  The  problem  must  be  sufficiently  well  defined  to  render  it  amenable  to 
decomposition  and  analysis. 

•  A  well-represented  group  of  interdisciplinary  specialists  must  be  actively 
involved  throughout  the  modeling  effort 

•  Systematic  data  assimilation,  diagramming,  and  documentation  meth  .  es 
must  be  established  at  the  inception  of  the  project. 

•  A  clear  understanding  of  how  much  detail  the  n.odel  will  contain  must  be 
established  at  the  beginning  of  the  project. 

•  A  clear  understanding  of  the  scope  of  the  model  must  be  established. 

•  A  clear  understanding  of  the  purpose  of  the  model  must  be  established. 

The  systems  analysis  approach  to  studying  design  decisions  in  engineering  activity 
is  a  top-down  examination  of  the  design  environment  and  the  elements  that  influence  the 
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environment.  Each  of  the  functional  components  of  the  engineering  design  process  are 
identified  and  studied  in  detail  with  respect  to  the  traditional  systems  analysis  architecture 
(inputs,  process,  and  outputs).  The  resulting  models  make  it  possible  to  study  alternative 
actions  during  design. 

1 .  Representative  Research  Studies 

This  section  describes  two  very  diverse  projects  that  employed  systems  analysis 
techniques  to  understand  design  decision  processes  in  the  software  engineering  domain 
[Ref.  12]  and  in  the  aerospace  domain  [Ref.  13]. 

a.  Love 

Love  [Ref.  12]  led  a  research  project  at  the  Space  Division  of  General  Electric  to 
study  the  software  development  process.  The  goal  of  this  project  was  to  establish  and  use 
a  framework  for  systematically  examining  and  understanding  the  process  of  computer 
program  development.  The  approach  adopted  was  to  identify  and  catalog  (with  respect  to 
the  framework)  variables  that  would  significantly  influence  the  development  process.  The 
aim  of  the  study  was  to  consider  all  human  and  computer  activities  that  contribute  to  the 
problem-solving  process. 

The  framework  established  for  this  study  was  patterned  after  the  classical  systems 
analysis  input-process-output  architecture.  Inputs  to  the  process  were  identified  as  task 
demands.  The  key  process  under  examination  was  identified  as  the  program  development 
process.  The  output  from  this  process  was  identified  as  the  operating  program. 

The  researchers  characterized  the  task  demands  as  both  implicit  and  explicit 
constraints  on  the  problem.  Forms  of  constraints  include  descriptions  of  the  problem 
(these  may  vary  from  detailed  statements  to  general  and  vague  descriptions,  and  they 
typically  change  during  the  course  of  software  development),  solution  specifications,  and 
resources  available  (these  include  the  quantity  and  quality  of  personnel,  hardware, 
software,  and  time  allocated).  The  character  of  these  constraints  will  significantly  affect  the 
software  development  process. 

The  process  itself  was  characterized  in  terms  of  four  elements; 

•  Programmer 

•  Computer  system 


Work  environment 
Software  development  system. 


The  researchers  attempted  to  describe  how  each  of  these  elements  influenced  the 
effectiveness  of  the  software  development  process.  To  assess  the  potential  effect  a 
programmer  may  have  on  the  process,  three  classes  of  factors  were  identified: 

•  Intellectual  capabilities  (cognitive  processing  capabilities  and  physiological 
characteristics) 

•  Experiences  (social,  educational,  and  work) 

•  Motivations  (physical,  emotional,  and  intellectual). 

To  assess  the  effect  a  computer  system  or  systems  would  have  on  the  efficiency  of 
programmers,  three  factors  were  proposed: 

'  Machine  efficiency  (speed,  capacity,  and  reliability) 

•  Mode  of  operation  (hardware  and  software  aspects  of  the  user  interface) 

•  Cost  of  using  the  system  (overall  costs  of  a  system  and  relative  costs  of 
performing  certain  tasks). 

These  factors  are  difficult  to  characterize  as  each  is  complex  and  potentially  multi¬ 
dimensional. 

The  third  factor  contributing  to  the  effectiveness  of  the  development  process,  the 
work  environment,  was  characterized  by  the  following  factors  and  sub-factors: 

•  Organizational 

-  Type  and  function  of  the  organization 

-  Type  of  work  within  the  organization 

-  Organization's  reward  structure 

•  Social 

-  Programmers'  attitudes  towards  work 

-  Programmers'  attitudes  towards  co-workers 

•  Physical  factors 

-  Atmosphere 

-  Auditory  environment 

-  Visual  environment. 
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The  final  factor  identified  as  a  major  influence  on  the  software  development 
process,  the  software  development  system,  was  characterized  in  terms  of  the  following 
four  factors; 

•  Program  design  (determining  a  representation  for  the  problem  to  be  solved) 

•  Program  notation  (procedures  for  coding  and  documenting  computer 
programs) 

*  Program  correcmess  (proving  that  an  algorithm  is  correct) 

♦  Program  verification  (verifying  the  algorithm's  representation  in  a  particular 
programming  language). 

This  research  project  provides  a  framework  for  examining  the  environment  (and 
influences)  in  which  design  occurs.  In  defining  this  framework,  the  researchers  attempted 
to  consider  the  entire  software  engineering  process,  with  particular  emphasis  on  "the 
individual  and  composite  effects  of  the  incredible  array  of  variables  which  affect  the 
[process]."  This  genre  of  framework  is  appropriate  for  any  engineering  activity,  regardless 
of  discipline.  The  value  of  the  framework  lies  in  its  use  as  a  tool  for  structuring  raw  data 
gleaned  from  other  studies,  such  as  those  considered  in  the  remaining  sections  of  this 
report  (obs^rvation/participant,  retrospective  analysis,  protocol  analysis,  and 
computational). 

b.  Fulton  and  Salley 

A  series  of  preliminary  studies  to  support  the  definition  of  hardware  and  software 
requirements  for  NASA's  Integrated  Programs  for  Aerospace  Design  (EPAD)  program  was 
conducted  during  the  1970s.  The  purpose  of  these  studies  was  to  formally  document  the 
aerospace  design  process.  Resulting  models  of  the  design  process  were  intended  to  serve 
as  source  material  for  the  requirements  definition  for  the  IPAD  software  as  described  by 
Fulton  and  Salley  [Ref.  13]. 

The  first  round  of  studies,  the  pre-contract  feasibility  studies,  were  conducted 
independently  by  Boeing  and  General  Dynamics.  Boeing  adopted  a  functional  perspective 
to  examine  the  product  development  process  for  three  types  of  vehicles— a  large  subsonic 
transport,  a  supersonic  transport,  and  a  hydrofoil.  These  examinations  resulted  in  "a 
detailed  picture  of  the  process  as  a  multilayered  network  of  functions."  General  Dynamics 
examined  the  design  process  from  the  perspective  of  individual  designers.  This  resulted  in 
a  definition  of  design  tasks,  interfaces,  and  information  flow.  The  net  result  from  both 
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efforts  was  "a  gross  characterization  of  the  computer-aided  design  capabilities  needed  to 
support  future  product  developments." 

After  the  official  IPAD  project  was  established  in  1976  with  a  contract  awarded  to 
Boeing  Commercial  Airplane  Company,  a  Design  Process  Definition  Phase  ensued.  The 
authors  describe  the  work  during  this  period  as  follows: 

The  approach  taken  was  to  organize  a  multidisciplinary  design  team  to 
conduct  a  systems  analysis  of  the  development  process  for  a  recent 
aerospace  product,  a  Boeing  747  commercial  air  transport.  The  team 
prepared  detailed  logic  charts  of  all  stages  of  design,  characterizing  design 
level,  the  major  engineering  steps  within  these  levels,  interfaces  among 
steps,  design  iterations,  and  analysis  and  design  computer  programs  used  to 
support  this  work.  1  he  747  study  was  subsequently  expanded  to  consider 
other  vehicles  including  a  future  advanced  aircraft  (SST),  military  aircraft 
(fighter),  and  a  non-aircraft  vehicle  (hydrofoil). 

The  system  analysis  work  was  also  broadened  to  investigate  design 
interfaces  to  manufacturing  and  where  major  data  exchanges  take  place. 

This  aspect  of  the  study  was  limited  to  design/manufacturing  interfaces 
since  the  ICAM  program  was  well  underway  to  investigate  technology 
associated  with  detailed  manufacturing  processes.  The  studies  did, 
however,  include  an  investigation  of  how  an  engineering  development  was 
managed  through  schedule  and  resource  control,  tracking,  and  assessment 
processes.  [Ref.  13] 

The  models  of  the  aerospace  design  process  were  created  through  the  use  of  a 
systems  analysis  technique  called  the  Systematic  Activity  Modeling  Method  (SAMM). 
Using  this  methodology,  the  design  process  was  depicted  in  a  network  structure,  with  each 
node  representing  related  activities  connected  to  other  nodes  by  information  flows  and 
feedback.  This  methodology  emphasizes  the  importance  of  modeling  the  data  flow  among 
the  activities  of  the  design  process.  SAMM  and  other  classical  data  modeling 
methodologies  are  reviewed  and  evaluated  in  the  accompanying  EDA  report.  Managing 
Engineering  Design  Information  [Ref.  11]. 

The  IPAD  modeling  effort  is  significant  because  it  "provided  a  system  level 
description  of  a  representative  aerospace  design  process,  the  first  such  description  available 
in  the  literature."  [Ref.  13] 


2 .  Strengths  and  Weaknesses  of  the  Approach 


a.  Strengths 

•  The  design  process  is  modeled  in  a  systematic  and  logical  manner.  The 
modeling  process  enables  the  identification  of  activities,  requirements, 
constraints,  enabling  mechanisms,  data  flows,  decisions,  informational 
interfaces,  and  relationships. 


This  approach  models  complex  design  environments  "in  such  a  way  that  one 
can  study  part  of  the  [environment]  at  a  time  while  not  losing  the  overall 
context."  [Ref.  14] 

"Formal  models  are  useful  for  doing  further  analysis  to  see  if  the  description  of 
a  [design  environment]  is  correct  and  consistent"  [Ref.  14] 


The  resultant  functional  diagrams  serve  as  a  tool  for  interdisciplinary 
communication,  meeting  control,  design  reviews,  etc. 

Models  produced  as  a  result  of  systems  analyses  are  tools  for  addressing  the 
following  questions: 

What  are  the  interrelationships  among  constituent  subprocesses? 

(How  does  the  behavior  of  one  subprocess  affect  others?) 

What  functions  (or  subprocesses)  have  common  elements  (such  as 
common  components)  or  common  data  flow? 

How  do  the  various  team  members  affect  the  processes  identified? 

Are  there  any  redundant  or  conflicting  subprocesses? 

Is  the  information  flow  reasonable? 

Is  there  unnecessary  information  provided  to  certain 
subprocesses? 

Is  there  any  missing  information  required  by  certain  subprocesses? 

Is  the  process  needlessly  complicated? 

Can  any  aspects  of  the  process  be  streamlined? 

Are  there  any  major  problems  in  the  design  process? 

What  are  the  boundaries  of  particular  subprocesses? 

The  modeling  process  is  highly  iterative  and  becomes  a  valuable  experience  for 
enhancing  communication  and  cooperation  among  team  players. 
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b.  Weaknesses 

•  This  approach  focuses  on  static  aspects  of  design  processes  and  is  sequential  in 
nature;  it  does  not  capture  behavior  patterns  that  emerge  over  time  or  temporal 
information  inherent  in  concurrent  processes. 

•  The  resulting  functional  diagrams  are  typically  very  large,  detailed,  and 
difficult  to  read.  The  terms  used  on  the  diagrams  are  usually  defined 
ambiguously  or  vaguely,  if  at  all,  so  understanding  the  diagrams  requires 
complete  understanding  of  the  terms  and  graphics  used.  Typically,  only  those 
actively  involved  in  the  development  of  the  model  can  read  or  make  use  of  it. 

•  The  resulting  models  usually  do  not  include  the  assumptions  they  are  based  on. 

•  The  human-machine  interface  area  is  usually  neglected.  [Ref.  15] 

•  The  resulting  models  cannot  be  verified  for  completeness  or  consistency. 

B  .  PARTICIPANT  OBSERVATION  STUDIES  OF  DESIGN  ACTIVITY 

Participant  observation  studies  (also  referred  to  as  informal  reporting  or  direct 
observation),  a  well-established  research  methodology  in  the  social  sciences,  are  gaining 
acceptance  within  the  engineering  design  research  community.  Although  an  informal 
methodology,  researchers  are  beginning  to  regard  this  approach  as  a  viable  means  of 
gathering  data  to  answer  questions  that  are  highly  context-dependent.  The  participant 
observation  methodology  offers  a  flexible  and  dynamic  approach  for  examining  the 
multidisciplinary  and  multifaceted  nature  of  the  design  process.  Meister  explains  the  need 
for  this  type  of  approach: 

It  is  impossible  to  understand  design,  and  in  particular  how  behavioral 
inputs  are  handled  in  design,  without  examining  its  sociocultural  context. 

The  role  the  engineer  plays  vis-a-vis  his  superiors  (the  management)  and 
other  contributors  to  the  design  process  has  significant  implications  for 
design  efficiency.  [Ref.  16] 

The  participant  observation  method  basically  consists  of  three  dimensions: 

•  The  design  activity  is  carefully  observed  within  the  industrial  design 
organization. 

•  The  researcher  participates  in  the  activity. 

•  All  observations  are  recorded  and  all  temporary  and  final  artifacts  of  the 
process  are  preserved. 


Useful  descriptions  of  participant  observation  studies  are  provided  by  design 
researchers  who  have  employed  this  approach.  Wallace  simply  describes  participant 
observation  studies  as  those  in  which  "the  researcher  actually  participates  in  the  design 
process,  both  contributes  to  it  and  observes  it  at  the  same  time"  [Ref.  17].  Bucciarelli 
describes  this  approach  as  a  "naturalistic,  observational  method  of  study,  yielding 
descriptions  of  complex  phenomena"  [Ref.  18].  He  explains; 

In  order  to  better  understand  the  engineering  design  process,  I  go  into  the 
firm  much  as  an  ethnographer  might  enter  a  remote  village  to  study  an 
aboriginal  culture.  I  not  only  observe  but  also  participate  as  an  engineer  in 
the  day-to-day  work  of  design  and  in  this  way  my  study  differs  from  that  of 
the  anthropologist  far  from  home.  I  join  the  world  of  participants  in  design, 
a  world  that  at  first  strikes  me  as  chaotic,  but  then  with  time,  settles  into  a 
more  comfortable  but  still  complex  process. 

The  raw  data  collected  in  this  manner  is  used  to  compile  analytical  descriptions  of 
the  design  activity.  These  descriptions  are  not  only  broad  in  scope  but  also  typically  very 
detailed.  Furthermore,  because  the  data  is  organization-  and  product-specific,  the  resulting 
descriptions  are  also. 

The  successful  application  of  participant  observation  research  techniques  requires, 
at  a  minimum,  the  following  three  conditions  to  be  met: 

•  The  practice  of  a  systematic  design  methodology  by  the  design  team  (i.e.,  Pahl 
and  Beitz).  This  provides  coherent  structures  for  categorizing  data  and 
comparing  theory  and  practice.  [Ref  17] 

•  Participation  by  the  researcher  from  the  inception  of  the  design  project  to  the 
fielding  of  the  product. 

•  "Large  measure  of  support  and  cooperation  from  the  design  team."  [Ref  17] 

With  respect  to  studying  design  decision  processes  at  an  organizational  level  of 
activity,  Simon  makes  a  number  of  interesting  remarks : 

Decision  making  in  organizational  settings,  which  is  much  less  well 
understood  than  individual  decision  making  and  problem  solving,  can  be 
studied  with  great  profit  using  already  established  methods  of  inquiry, 
especially  through  intensive  long-range  studies  within  individual 
organizations.... 

Although  the  decision-making  processes  of  organizations  have  been  studied 
in  the  field  on  a  limited  scale,  a  great  many  more  such  intensive  studies  will 
be  needed  before  the  full  range  of  techniques  used  by  organizations  to  make 
their  decisions  is  understood,  and  before  the  strengths  and  weaknesses  of 
these  techniques  are  grasped....  [Ref  10] 
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The  projects  described  in  the  following  section  represent  two  such  intensive  and 
valuable  studies  into  engineering  design  decision  processes  using  the  participant 
observation  approach. 

1 .  Representative  Research  Studies 
a.  Wallace 

Wallace  [Ref.  17]  is  involved  in  a  design  research  program  at  the  University  of 
Cambridge  which  focuses  on  how  the  performance  of  design  teams  can  be  improved.  As 
part  of  this  research  program,  a  "detailed  participant  observation  study  of  an  engineering 
design  project  in  industry"  was  initiated  in  1982.  The  goal  of  this  study  was  to  "carry  out 
an  exploratory  investigation  into  the  research  methods  and  likely  outcomes  of  studying  a 
team  working  through  the  engineering  design  process  in  industry." 

The  approach  adopted  for  this  study  consisted  of  participant  observation  methods  to 
collect  and  analyze  data  generated  throughout  the  design  process.  Raw  data  collected 
during  the  study  included  field  notes,  correspondence,  design  notes  and  drawings,  weekly 
reports,  technical  reports,  audio  tape  recordings,  and  case  histories  from  other  projects. 

Prior  to  massaging  this  raw  data,  the  researchers  found  that  a  model  was  needed  to 
organize  the  data  that  was  being  collected.  As  the  author  explains; 

During  the  Cambridge  project,  many  of  the  available  models  of  the  design 
process  were  reviewed,  but  none  was  entirely  satisfactory  for  structuring 
and  analyzing  the  data.  Using  the  existing  models  as  a  starting  point,  a  new 
diagrammatic  model  was  developed  which  set  the  design  process  in  a  broad 
industrial  context  and  identified  five  hierarchical  levels  of  resolution.  This 
provided  a  satisfactory  framework  for  analyzing  the  data  from  the  particular 
design  project  studied,  but  whether  it  is  valid  for  other  types  of  projects  is 
impossible  to  say  at  this  stage.  [Ref.  17,  Figure  II- 1] 

This  model  is  unique  because  it  distinguishes  between  the  overall  project  effort  and 
the  engineering  design  process.  It  was  designed  to  be  used  to  classify  inputs  to  the  projects 
at  different  levels  within  the  context  in  which  engineering  design  occurs. 

Essential  elements  from  the  raw  data  were  compiled  into  interchange  data  sheets, 
which  were  subsequently  transferred  to  a  microcomputer  data  base  and  analyzed  by  means 
of  spreadsheet  calculations.  From  this  data  base,  the  researchers  were  able  to  generate 
analytic  reports  and  graphs  that  described  elements  such  as  the  following: 
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•  Each  participant's  work  effort 

•  Amount  of  time  spent  on  the  different  phases  of  design 

•  Amount  of  time  spent  in  meetings 

•  Use  of  methods  and  aids 

•  Ehstribution  of  work  effort  completed  in  various  locations 

•  Quantified  measures  of  subjective  data  on  the  mood  of  participants. 

These  quantitative  analyses  described  the  design  project  in  terms  of  the  activities 
and  outputs  produced  but  did  not  explain  the  reasons  behind  the  process.  To  address  this 
issue,  the  researchers  attempted  to  identify  influences  that  affected  the  project.  The 
influences  were  characterized  with  respect  to  the  five  levels  of  resolution  defined  in  the 
context  model  and  were  characterized  in  terms  of  observed  contributing  factors.  With  this 
framework,  the  researchers  were  able  to  examine  which  influences  [Figure  II- 1]  had  the 
greatest  observed  effect  on  the  design  process.  These  examinations  led  to  an  attempt  to 
profile  the  project  according  to  the  effect  of  the  influences. 

With  this  myriad  of  derived  information,  the  researchers  were  able  to  characterize 
the  project  in  terms  of  quantitative  and  qualitative  measures.  The  analytical  reports, 
analytical  graphs,  narrative  case  histories,  and  influence  charts  provided  the  means  to  attain 
a  deeper  understanding  of  the  activities,  outputs,  and  influences  of  the  engineering  design 
process. 

b.  Bucciarelli 

Bucciarelli  [Ref.  18]  is  conducting  research  at  the  MIT  School  of  Engineering  to 
develop  a  better  understanding  of  the  engineering  design  process.  He  is  specifically 
interested  in  exploring  "the  social  as  well  as  technical  elements  of  decisions  made  in  a 
firm's  endeavor  to  develop  and  market  its  wares."  To  do  this,  a  long-range  participant 
observation  study  was  initiated  in  1982.  The  principle  objective  of  this  study  was  to 
"determine  how  values  inform  decisions  and  thus  affect  the  forms  of  technology  that 
emerge  from  that  process." 


Level  of 
Resolution 

Influence  Category 

Examples  of  Contributing  Factors 

Macroeconomic 

(Environment) 

External 

Social/Political/E  conomic/Ecological/Legal/Random 

Microeconomic 

(Market) 

Market 

Resource  Availability 
Customer 

Demand/Competition/Risk/Finance/Services/People 
Need/U  rg  ency /Ex  pectatio  ns 

Corporate 

(Company) 

Corporate  Structure 
Corporate  Systems 
Corporate  Strategy 

Shared  Values 
Management  Style 
Management  Skills 
Management  Staff 

Size/Span/Complexity/Integration/Remuneration 
Objectives/Risk-Taking/Commitment/Enthusiasm 
Autocratic/Benevolent/Coordinating/Resource  Use 
Judgment/Confidence 

Project 

Design  Task 

Design  Team 

Design  Techniques 

Design  Output 

Magnitude/Complexity /Risk/Novelty /Quantity /Timing 
Expertise/Experience/User/Role  Balance/Motivation 
Systematic  Procedures/Communicating/Motivating 
Productivity /Quality 

Personal 

Personal  Knowledge 
Personal  Skills 

Personal  Attitude 

Personal  Movitation 
Personal  Output 

Knowledge  Base/Usefulness/Perception 

Imagination/Self-Discipline/Integrity/Enthusiasm 

Involvement/Productivity/Quality 

Figure  11*1.  Design  Process  Infiuences 


Throughout  the  course  of  this  study  a  vast  amount  of  raw  data  is  being  collected, 
including;  # 

•  Field  notes  (conversations;  sketches;  informal  meeting  proceedings;  language 
used;  gestures;  sources  of  problems,  interruptions,  and  delays;  and  other 
observations  of  daily  activity) 

•  Design  review  notes  • 

•  Formal  engineering  drawings 

•  Managerial  charts 

•  Product  specification  sheets  ^ 

•  Employee  newsletters 

•  Formal  interviews  with  designers 

•  Audio  tape  recordings  of  designers'  ideas  and  impressions. 
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Bucciarelli  is  in  the  process  of  developing  a  comprehensive  framework  for 
analyzing  the  raw  data.  The  results  of  this  analysis  will  be  a  model  of  the  design  process  in 
terms  of  the  "the  role  of  tradition,  habit,  and  values  in  technical  discourse  and  decision 
making."  He  is  working  with  two  different  forms  of  representations-narrative  case  studies 
and  topical  classification  schemes  to  structure  the  data.  With  these  models,  he  intends  to 
reveal  "the  character  and  consequences  of  the  discourse  through  which  design  participants 
communicate  with  each  other  under  conditions  of  sustained  uncertainty  and  ambiguity." 
These  descriptions  of  the  design  process  will  explain  such  observations  as: 

...different  participants  in  the  design  process  have  different  perceptions  of 
the  design. ..The  task  of  design  is  then  as  much  a  matter  of  getting  different 
people  to  share  a  common  perspective,  to  agree  on  the  most  significant 
issues,  and  to  shape  consensus  on  what  must  be  done  next,  as  it  is  a  matter 
of  concept  formation,  evaluation  of  alternatives,  costing  and  sizing.... 

[Ref.  18] 

2 .  Strengths  and  Weaknesses  of  the  Approach 

a.  Strengths 

•  The  participant  observation  approach  "provides  the  [resercher]  opportunity  to 
experience  the  design  activity  and  increases  the  chances  of  being  aware  of  the 
many  subtle  factors  and  influences  which  affect  the  cor  of  the  project.  It 
also  provides  a  limited  opportunity  to  try  out  some  ideas  and  methods.  For 
instance,  a  particular  design  approach  can  be  used  to  coordinate  the  design 
team  and  influence  its  progress."  [Ref.  17] 

•  This  approach  is  useful  for  eliciting  information  that  is  difficult  to  express. 

•  This  approach  highlights  the  complexity  of  the  human  interactions  involved  in 
the  design  process.  [Ref.  18] 

•  "Participant  observation  offers  the  potential  for  gathering  a  very  large  amount 
of  quantitative  and  qualitative  data,  and  it  provides  the  opportunity  for  testing 
out  a  limited  number  of  ideas  and  methods."  [Ref.  17] 

•  The  data  that  is  collected  serves  as  sources  of  input  to  other  forms  of  empirical 
inquiry  such  as  computational  experiments.  [Ref.  19] 

b.  Weaknesses 

Wallace  provides  a  number  of  observations  from  the  study  he  conducted  regarding 
weaknesses  in  this  method.  These  include: 


•  ...with  participant  observation  the  problems  for  the  researcher  increase,  and 
include: 

( 1 )  Gaining  the  complete  confidence  of  the  design  team; 

(2)  Splitting  the  time  between  'participating'  and  'observing'; 

(3)  Maintaining  a  neutral  viewpoint  and  keeping  the  observations  impartial 
while  being  closely  involved  in  the  project. 

•  It  falls  short  of  a  planned  experiment.  [Necessary  to  systematically  study  a 
particular  theory] 

•  The  procedure. ..is  very  laborious. 

•  Every  hour  spent  recording  data  required  at  least  another  hour  to  write  up  and 
reference  the  data. 

•  Difficulties  arose  during  data  collection,  processing,  and  analysis.  [Ref  17] 

Additional  weaknesses  identified  include 

•  Recording  anything  other  than  what  is  written  down  or  retained  in  other 
tangible  forms  is  difficult.  [Ref.  20] 

•  One  cannot  replay  the  design  episode  to  do  a  detailed  analysis  of  what 
occurred;  only  a  momentary  analysis,  while  one  is  observing  and  participating, 
is  possible. 

C.  RETROSPECTIVE  ANALYSIS 

The  examination  of  retrospective  reports,  case  studies,  and  other  forms  of  historical 
research  is  a  traditional  and  universally  employed  approach  for  deriving  general  principles 
to  describe  complex  phenomena.  Accounts  of  past  events  from  a  suitable  variety  of 
perspectives  and  domains  provide  rich  sources  of  raw  data  for  drawing  generalizations 
about  the  nature  of  design.  With  this  data,  design  researchers  are  able  to  identify 
fundamental  factors  associated  with  both  successful  and  disastrous  design. 

The  type  of  information  that  can  be  gleaned  from  a  retrospective  report  depends  on 
the  purpose  of  the  report  and  the  manner  in  which  the  report  was  generated.  A  typical  case 
study,  prepared  for  historical  purposes,  describes  the  design  project  in  terms  of  discrete 
phases.  For  each  phase,  the  following  types  of  raw  data  may  be  provided: 

•  Task  descriptions 

•  Requirements  and  constraints 
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Predetermined  decision  points 
Manner  in  which  the  tasks  were  accomplished 
Expjertise  and  knowledge  required 
Technical  parameters  selected 
Parameter  dependencies 
Sizing  of  the  parameters 
Manufacturing  considerations 
Artifacts  of  the  design  process 
Analysis  tools  used 
Type  of  reviews  conducted 
Type  of  tests  conducted 
Subject  of  tests 

Design  and  manufacturing  interface 
Quality  assurance  requirements 
System  integration  requirements 
System  integration  tests 
Role  of  the  designers  in  the  field  support. 

With  this  raw  data,  the  following  types  of  information  may  be  produced: 
Patterns  in  design  processes  that  led  to  solutions 
Commonalities  in  design  processes  across  all  domains 
Problem  areas 
Important  design  decisions 
Effect  or  influence  of  decision  makers 
Reasons  for  the  decisions 
Information  used  to  make  design  decisions 
Evolution  of  the  project  knowledge  base 
Effect  of  internal  procedures 
Effect  of  organizational  constraints 

Effect  of  the  types  of  tasks  and  activities  conducted  in  the  past 
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•  Manner  in  which  the  product  evolved 

»  Characterizations  of  the  organizations 

•  Characterizations  of  the  people  involved 

•  Effect  of  external  influences 

•  Relationships  among  the  engineering  function,  management,  and  indirect 
personnel 

•  Relationships  among  internal  and  external  influences 

•  Effect  of  tools  and  techniques  employed. 

With  this  information,  analyses  of  case  studies  serve  as  a  basis  for  defining,  testing,  and 
refining  generic  models  of  the  design  process. 

Retrospective  reports  are  often  narratives  provided  by  key  individuals  involved  in 
the  design  project.  These  reports  cover  significant,  specific  aspects  of  the  project.  Other 
sources  of  historical  data  include  responses  to  structured  and  unstructured  interviews  or 
questionnaires,  project  documentation  (technical,  administrative,  and  managerial),  and  any 
other  informal  account  of  the  project  history  (such  as  press  releases).  Of  these  modes  of 
data  collection,  interviews  provide  the  most  flexibility  and  promise  of  attaining  the 
information  required.  As  explained  by  Magee  [Ref.  2]: 

Interviewers  can  make  sure  respondents  have  understood  any  question  and 
the  purpose  of  that  question.  They  can  probe  further  when  a  response 
warrants  it;  ask  the  expert  to  clarify  a  response  so  that  it  is  understood;  and 
pick  up  clues  from  the  expert's  non-verbal  communication  about  the 
dependability  or  otherwise  of  the  information.  Perhaps  more  importantly 
interviewers  can  buUd  up  a  rapport  with  experts  and  create  an  atmosphere  in 
which  they  find  it  easier  to  communicate... 

To  analyze  source  data,  a  framework  that  structures  the  data  in  terms  of  specific 
aspects  of  the  design  process  is  required.  Such  a  framework  serves  as  the  primary  tool  for 
answering  research  questions  about  design  processes.  The  framework  created  by  Waldron 
and  Waldron  [Ref.  7],  described  in  the  next  section,  is  an  excellent  example  of  the  manner 
in  which  retrospective  data  can  be  structured  to  identify  the  major  design  decisions  that 
were  made  during  the  design  of  a  complex  mechanical  system. 

Examining  the  design  process  a  posteriori  yields  valuable  insights  into  the  design 
process.  The  following  conditions,  when  met,  increase  the  effectiveness  of  retrospective 
analysis: 


•  The  design  project  must  be  well  documented,  including  frequent  reports  from  a 
variety  of  perspectives  covering  a  number  of  aspects  of  the  design  process. 

•  The  entire  project  team  must  be  available  to  answer  questions  about  the  project, 
including  key  individuals  and  others  who  directly  or  indirectly  affected  the 
design. 

•  Primary  and  secondary  issues  must  be  identifiable  from  the  source  material. 

•  All  historical  information  must  be  placed  in  a  time  sequence. 

1 .  Representative  Research  Studies 

In  this  section,  the  ways  in  which  various  researchers  have  examined  case  studies 
of  design  projects  to  understand  the  design  decision  process  is  presented. 

a.  Waldron  and  Waldron 

Waldron  and  Waldron  [Ref.  21]  are  conducting  research  at  Ohio  State  University 
into  the  conceptual  mechanical  design  process.  One  aspect  of  their  research  consisted  of  a 
retrospective  analysis  of  the  system  design  of  the  Adaptive  Suspension  Vehicle  developed 
in  the  Department  of  Mechanical  Engineering,  Ohio  State  University.  The  results  from  this 
study  are  intended  to  contribute  to  the  foundation  of  a  theory  of  the  mechanical  design 
process. 

The  research  methodology  used  in  this  study  involved  collecting  case  history 
information  and  formulating  a  model  of  the  design  decision  process  from  the  retrospective 
reports.  The  case  histories  were  narratives  (from  the  principle  engineer  of  the  project) 
describing  the  evolution  and  fmal  outcome  of  the  design  concepts  produced  throughout  the 
project.  The  principle  engineer  (and  others  involved  in  the  design)  were  asked  to  "recount 
the  state  of  the  knowledge  at  each  decision  point  [in  the  design  process]  including 
misleading  and  erroneous  knowledge." 

With  the  retrospective  narratives,  the  researchers  developed  a  model  of  the  design 
decision  process  for  this  particular  product  The  representation  is  in  the  form  of  a  chart  that 
shows  the  following: 

•  Major  events  in  the  design  process  (design  decisions,  subsystem  phases,  and 
other  events  affecting  the  design  process) 

•  Major  events  in  the  manufacturing  process 

•  States  of  the  knowledge  base  over  time 


•  Information  pathways 

•  Events  peripheral  to  the  design  and  manufacturing  processes  that  helped 
establish  an  environmental  timeframe. 

This  model  is  organized  chronologically  so  that  the  evolution  of  the  knowledge  bases  used 
to  make  design  decisions  and  the  progression  of  the  development  of  design  concepts  can  be 
examined  and  tracked. 


Using  this  model,  much  analysis  was  possible.  It  was  used  to  identify  issues 
involved  in  major  design  decisions  and  examine  the  way  decisions  were  made.  From  this 
analysis,  the  authors  were  able  to  draw  general  conclusions  about  the  design  process  of  this 
particular  product  which  led  to  a  theory  of  the  mechanical  design  process.  The  authors 
suggest  that  this  is  "a  reasonable  approach  for  identifying  an  organizational  structure  of  the 
design  process.. .[using]. ..major  design  decisions  as  the  organizing  principle”  [Ref.  21]. 
More  important,  the  model  provides  a  solid  foundation  for  the  analysis  of  data  generated  as 
the  result  of  other  methodologies  at  the  individual  level,  such  as  protocol  analysis  [Section 
III- A]. 


b.  Peterson,  Hagel,  Nadler,  and  Chignell 

Peterson,  Hagel,  Nadler,  and  Chignell  [Ref.  22]  are  conducting  research  at  the 
University  of  Southern  California  to  "identify  and  investigate  process  and  decision  aids  in 
design."  The  methodology  developed  for  this  project,  loosely  based  on  Nadler's  Timeline 
Scenario  Methodology,  is  called  Retrospective  Timeline  Scenario  Methodology.  This 
method  consists  of  conducting  carefully  planned  structured  interviews  in  which  the 
interview  questions  are  designed  to  elicit  comprehensive  information  describing  the 
activities  and  processes  that  occurred  during  a  successful  design  project.  The  case  study 
data  are  gathered  from  a  timeline  perspective  and  are  organized  in  a  temporal  manner.  The 
model  generated  is  a  timeline  scenario  that  "shows  relationships  between  various  elements 
(such  as  people,  information,  activities,  modes  of  communication,  and  critical  events)  over 
time  (time  is  the  X-axis)." 


While  results  from  the  initial  phase  of  the  study  are  under  formation,  some 
prehminaiy  findings  include: 

•  In  some  cases,  the  interviews  yielded  more  questions  than  answers. 

•  A  number  of  elements  of  design  processes  were  identified. 
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•  Established  corporate  procedures  and  large  hierarchical  design  teams  in  one 
case  resulted  in  overly  stringent  specifications,  which  effectively  stifled 
innovation. 

•  Corporate  design  procedures  and  ideologies  were  beneficial  when  management 
achieved  a  non-restrictive  design  atmosphere. 

•  Academic  stage  models  of  the  design  process  are  not  followed  in  practice. 
"Most  industrial  design  seems  to  be  a  series  of  relatively  unstructured 
activities,  which  eventually  (and  often  through  a  series  of  costly  and  time- 
consuming  iterations)  result  in  a  designed  product." 

•  The  data  do  not  support  the  notion  that  designers  divide  and  conquer  difficult 
design  problems  or  decompose  goals  into  subgoals. 

The  authors  conclude  from  this  study  that  "process  aids  in  conceptual  design  are 
difficult  to  identify."  Furthermore,  "preparing  designers  to  be  purpose-oriented,  tolerant  of 
ambiguity,  capable  of  working  with  others,  and  appreciative  of  soft  data  might  help 
improve  design  performance." 

This  methodology  provided  the  researchers  with  a  systematic  and  comprehensive 
framework  for  gathering,  organizing,  and  analyzing  real-world  data.  The  data  were  used  to 
identify  and  characterize  design  decisions  and  examine  both  positive  and  negative  effects  of 
decisions  made  during  the  design  process.  The  resultant  models,  the  project  timeline 
scenarios,  highlight  significant  information  about  the  contextual  and  temporal  aspects  of 
design. 


c.  Petroski 

Petroski  [Ref.  23],  from  the  Department  of  Civil  Engineering  at  Duke,  is  interested 
in  applying  knowledge  of  past  design  failures  to  the  design  decision  process  of  current 
design  projects.  Typically,  the  design  decision  process  is  influenced  by  experience  from 
and  knowledge  of  past  design  successes.  Petroski,  however,  feels  that  knowledge  of 
design  decisions  that  led  to  failures  is  essential  to  understand  the  design  process  more 
thoroughly  and  to  successfully  avoid  making  similar  errors. 

The  focus  of  the  study  reported  is  bridge  design.  Petroski  analyzes  the  history  of 
bridge  design  to  identify  design  decisions  concerning  determinations  such  as  choice  of 
material  or  design,  process.  He  provides  illustrations  of  specific  examples  of  situations  in 
which  the  lack  of  knowledge  about  previous  design  decisions  led  to  the  design  of 
subsequent  bridges  that  failed. 


Petroski  investigates  historical  examples  of  design  failures  to  understand  the  design 
decision  process  of  a  specific  engineering  domain  in  a  very  informal  manner.  This  study 
illustrates  the  value  of  historical  case  studies  as  rich  sources  of  data  for  developing  models 
of  design  decision  processes. 

d.  Promisel,  Hartel,  Kaplan,  Marcus,  and  Whittenburg 

Promisel,  Hartel,  Kaplan,  Marcus  and  Whittenburg  [Ref.  24]  of  the  Systems 
Research  Laboratory  at  the  US  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences,  conducted  case  studies  of  the  development  of  four  weapons  systems  (the 
STINGER,  Multiple  Launch  Rocket  System,  BLACK  HAWK,  and  Ml  Fault  Detection 
and  Isolation  Subsystems).  The  specific  focus  of  this  research  was  to  assess  the  extent  to 
which  human  factor  issues  were  considered  throughout  the  development  process  of  each 
weapon  system. 

The  methodology  used  during  this  study  was  called  reverse  engineering.  This 
approach  consists  of 

•  Defining  and  describing  the  system 

•  Reviewing  requirements  documents  and  determining  how  system  performance 
was  specified 

•  Analyzing  test  and  evaluation  data  and  comparing  the  data  to  performance 
criteria 

•  Identifying  problem  areas  in  system  performance 

•  Examining  human  factors,  manpower,  personnel,  and  training  factors  for  their 
effect  on  problematic  aspects  of  system  performance 

•  Reviewing  the  weapon  system  acquisition  process  to  identify  features  that 
contributed  to  human  factors  issues. 

To  analyze  the  wealth  of  life  cycle  data,  the  researchers  developed  a  matrix  that 
relates  descriptions  of  the  primary  activities  of  the  system  development  process,  functions 
within  the  weapon  system  acquisition  process,  and  acquisition  process  milestones.  Within 
this  structure,  they  were  able  to  "identify  causal  factors  contributing  to  [the  human  factors] 
deficiencies"  that  were  found.  The  matrix  is  a  general  framework  that  describes  the 
weapon  system  acquisition  process.  During  this  study,  it  was  used  to  synthesize  the  data 
from  the  four  case  studies  to  determine  how  the  weapon  system  acquisition  process  could 
be  improved. 
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This  research  illustrates  the  manner  in  which  data  can  be  systematically  extracted 
from  case  studies  and  synthesized  into  a  general  framework  to  study  aspects  of  the  design 
process  (or,  in  this  case,  the  weapon  system  acquisition  process). 

2 .  Strengths  and  Weaknesses  of  the  Approach 

a.  Strengths 

•  Allows  researchers  to  examine  real-world  practices.  [Ref.  22] 

•  Produces  information  that  could  improve  the  likelihood  of  design  success  in 
other  design  activities.  [Ref.  22] 

•  "Merges  advantages  of  the  think-aloud  method  with  a  definitive  set  of  broad- 
based  and  specific  questions  concerning  the  various  elements  and  aspects  of  a 
successful  real  world  design  which  spans  the  life  course  of  the  particular 
design  object."  [Ref.  22] 

•  "Interviews  illustrate  the  difficulties  designers  face."  [Ref.  22] 

•  "This  method  allows  for  the  uncovering  of  a  clear  portrayal  of  the  various 
elements  of  design  projects."  [Ref.  22] 

•  "The  method  works  best  with  a  large  number  of  cases,  and  may  be  a  useful 
design  project  characterization  since  it  does  show  changes  over  time."  [Ref. 
22] 

•  The  resultant  models  depict  relationships  between  the  diverse  elements  of  the 
design  decision  process. 

b.  Weaknesses 

•  "In  this  retrospective  or  historical  research,  information  is  gathered  in  the 
present  about  projects  which  have  finished.  As  the  events  can  never  be 
repeated,  the  researcher  has  to  rely  on  documented  evidence  and  the  memories 
of  those  involved  during  the  project.  Memories  can  be  unreliable  and  there  is 
always  the  tendency  to  modify  and  dramatize  the  events  of  the  past,  in  other 
words,  the  quality  of  the  data  is  often  poor.  Not  surprisingly,  case  histories 
often  tend  to  adopt  a  specific  viewpoint  and,  because  of  the  way  the  results  are 
structured,  give  the  impression  that  the  design  process  was  extremely  well 
ordered  and  logical."  [Ref.  17] 

•  Testing  hypotheses  using  retrospective  case  studies  is  very  difficult.  [Ref.  22] 

•  Some  designers  are  unable  to  recount  how  a  new  product  was  invented.  [Ref. 

22] 
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•  "With  retrospective  histories,  designers  only  recall  major  or  critical  events,  and 
not  necessarily  what  transpired  to  give  rise  to  them.. .Designers  cannot 
remember  all  the  considerations  in  design  decisions;  at  best,  they  can  only  tell 
you  what  was  the  decision."  [Ref.  22] 

•  A  strong  element  of  subjectivity  exists  in  the  interviewing  process.  [Ref.  21] 

•  It  is  difficult  to  verbalize  reasoning  that  is  spatial  in  nature  or  that  involves 
visualization  of  the  working  of  the  system.  [Ref.  21] 

D.  KNOWLEDGE  REPRESENTATION 

Knowledge  representation  techniques,  developed  in  response  to  an  increasingly 
high  level  of  interest  in  encoding  and  manipulating  domain-specific  expertise  (especially  in 
the  domain  of  engineering  design),  are  invaluable  for  studying  design  at  both  individual 
and  organizational  levels  of  activity.  This  section  reviews  the  manner  in  which  knowledge 
representation  schemas  enrich  understanding  of  organizational  level  design  decision 
processes. 

The  purpose  of  knowledge  representation  schemas  is  to  capture  knowledge 
(information  with  an  understanding  of  how  it  may  be  applied)  in  a  form  that  can  be 
processed  by  a  machine.  This  is  a  particularly  challenging  quest  for  the  engineering 
domain  as  design  data  at  any  point  during  the  product  life  cycle  are  typically  characterized 
as  follows; 

•  Complex  (many  specified  and  unspecified  relationships  among  design  objects, 
attributes,  and  relationships) 

•  Incomplete  (Any  portion  of  the  design  at  any  point  may  be  only  partially 
described.) 

•  Inconsistent  (Attributes,  relationships,  and  objects  may  contradict  one  another 
on  different  levels;  there  may  be  multiple  representations  of  the  same  object  or 
component.) 

•  Ambiguous  (Attributes,  relationships,  and  objects  may  be  created  with  much 
implicit  information  that  is  never  explicitly  included  in  the  product  design 
definition.) 

•  Hierarchical  (The  artifact  and  components  of  the  artifact  are  described  at  many 
different  levels  of  abstraction,  each  of  which  may  or  may  not  be  complete  and 
consistent  with  one  another.) 

•  Dynamic  (changes  over  time). 


Furthermore,  engineering  design  data  appears  in  a  myriad  of  forms,  including 
parametric,  tabular,  geometrical,  heuristic,  matrix,  and  textual.  Engineering  design  data 
(data  describing  the  final  product  and  the  requisite  knowledge  used  to  create  it)  are 
fundamentally  different  from  the  traditional  types  of  data  handled  by  a  computer,  such  as 
business  data.  (Details  of  this  difference  are  given  in  the  accompanying  report.  Managing 
Engineering  Design  Information  [Ref.  11].) 

A  simplified  view  of  the  design  process  is  necessary  to  begin  to  grapple  with  the 
knowledge  representation  problem  posed  by  engineering  design.  A  description  of  a  basic 
structure  of  design  suggested  by  Lindhult  is  fairly  representative  of  those  shared  by  a 
number  of  design  researchers  [Ref.  25]; 

The  design  process  can  be  viewed  simplistically  as  comprising  the 
following  series  of  steps: 

Problem  Definition 
Conceptual  Design 
Design  Development 
Design  Documentation 
Construction  and  Evaluation 

The  entire  design  process  is  a  series  of  iterative  cycles  composed  of 
dissimilar  activities  which  range  from  intuitive  leaps  to  complex  technical 
calculations.  Within  each  of  the  above  steps,  information  is  analyzed  and 
synthesized  by  the  designer  and  a  physical  form  begins  to  evolve. 

Frameworks  such  as  this  provide  the  basis  necessary  to  apply  Newell  &  Simon's 
information  processing  theory  of  human  problem  solving  to  the  task  of  engineering  design. 
Newell  and  Simon's  theory  of  information  processing  describes  a  problem  in  terms  of 
states  .  The  solution  to  a  problem  is  achieved  by  the  progressive  application  of  operators, 
which  transform  one  problem  state  into  a  different  state  [Ref.  7].  This  approach  requires 
an  implementation  of  a  task  environment  (a  search  space),  which  is  a  formal  specification 
of  the  knowledge  needed  to  solve  a  problem  (or,  in  this  case,  design  an  artifact).  A 
knowledge  representation  schema,  a  domain-independent  generalized  strategy  for  encoding 
knowledge,  is  used  to  model  a  particular  task  environment.  Through  this  method,  specific 
implementations  of  schemas  will  contain  domain-specific  knowledge. 


1 .  Representative  Research  Studies 

The  following  projects  describe  different  approaches,  currently  under  development, 
for  building  representations  of  task  environments.  The  resulting  design  spaces  are  useful 
models  of  design  decision  structures. 

a.  Kurumatani  and  Yoshikawa 

At  the  University  of  Tokyo,  Kurumatani  and  Yoshikawa  [Ref.  26]  are  developing  a 
methodology  for  modeling  knowledge  of  design  objects  and  processes.  This  methodology 
is  based  on  a  formal,  axiomatic  theory  of  design.  General  Design  Theory,  proposed  by 
Yoshikawa.  General  Design  Theory  provides  axioms  for  characterizing  human  design 
activity.  From  these  axioms,  the  following  assertions  regarding  the  representation  of 
design  knowledge  are  derived: 

•  Any  entity  can  be  described  by  its  properties,  especially  by  its  attributes. 

•  Any  entity  is  treated  as  being  on  equal  terms  with  other  entities—all  of  the 
entities  should  be  described  in  the  same  way  without  assuming  any  absolute 
hierarchy  of  entities. 

•  Design  process  can  be  formalized  as  the  operation  of  abstract  concepts. 
[Ref.  26] 

The  authors  propose  a  schema  based  on  the  General  Design  Theory  for  representing 
design  objects  and  design  processes.  In  this  schema,  design  objects  are  characterized  by 
their  properties  and  structure  (both  internal  and  external).  This  provides  the  means  for 
representing  the  function,  attributes,  and  relationships  to  other  objects.  With  this  schema,  a 
machine  can  be  defined  as  a  network  of  objects. 

Design  processes  are  defined  as  processes  "in  which  the  design  object  is  detailized 
by  making  intersections  of  abstract  concepts."  Design  processes  are  a  series  of 
transformations  of  design  objects  during  which  the  the  design  object  concepts  become 
increasingly  refined.  These  transformations  are  enabled  by  access  to  information  belonging 
to  the  design  objects.  This  view  of  design  suggests  "the  possibility  that  a  design  process 
can  be  explained  as  the  alternate  actions  of  two  op>erators  to  the  design  object"  [Ref.  26] 

This  schema  is  being  implemented  in  a  frame-based  system  by  the  authors. 
Because  it  is  based  on  a  theoretic  model  of  design,  yet  provides  a  practical  approach  for 
representing  design  knowledge  symbolically,  the  schema  is  especially  interesting. 


b.  Kaiay,  Swerdloff,  and  Harfmann 
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Kaiay,  Swerdloff,  and  Harfmann  [Ref.  26]  are  developing  and  implementing 
"fundamental  paradigms  of  design  as  a  computable  process"  at  the  State  University  of  New 
York  (SUNY)  at  Buffalo.  The  focus  of  this  research  is  the  exploration  of  the  potential  role 
of  computers  as  design  partners,  rather  than  merely  design  tools,  to  increase  the  flexibility 
and  effectiveness  of  design  processes.  The  basic  premise  underlying  the  paradigms  is  that 
a  design  process  is  a  search  for  a  "physical  or  organizational  scheme  which  when  realized 
will  achieve  certain  goals  and  abide  by  certain  constraints."  To  experiment  with  this 
concept,  the  authors  are  "providing  computers  with  knowledge  similar  to  that  of  designers 
stored  in  the  form  of  performance,  goals,  and  design  plans."  [Ref.  27] 

The  basis  for  the  research  is  Newell  and  Simon's  state/transition  model  of  problem¬ 
solving  (see  Section  D.l),  in  which  states  represent  design  goals  in  various  levels  of 
abstraction. 

A  design  goal  is  a  "context  dependent  constraint  which  defines  conditions 
that  candidate  solutions  must  meet."  Goals  are  organized  in  a  goal 
hierarchy.  "Transitions  represent  the  means  for  choosing  and  achieving 
goals."  Transitions  between  goals  are  achieved  under  the  control  of  an 
overall  design  p'an,  portions  of  which  are  carried  out  by  the  designer  and 
the  system.  [Ref.  27] 

The  authors  have  implemented  the  goal  hierarchy  and  a  design  process  control 
mechanism  in  Prolog,  and  goals  are  represented  in  frames.  These  structures  contain 
descriptive  knowledge  (constraints)  and  procedural  knowledge  (weightings  for  preferred 
solutions). 

This  computational  paradigm  of  design  is  unique  because  it  allows  the  designer  to 
determine  dynamically  which  tasks  will  be  performed  (which  decisions  will  be  made)  by 
the  system  and  which  will  be  made  by  the  designer.  The  structure  of  the  decisions  (the 
goal  hierarchy)  and  knowledge  about  design  (the  content  of  the  goal  frames)  are  maintained 
by  the  system.  The  actual  execution  of  the  design  decision  process  is  controlled  by  the 
designer.  The  authors  suggest  that  this  design  paradigm  "is  applicable  to  decision-making 
processes  of  many  kinds."  Theoretically,  the  knowledge  base  (which  currently  contains 
architectural  engineering  expertise)  can  be  populated  with  any  domain-specific  knowledge. 
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c.  McCall 


McCall  [Ref.  28]  is  developing  a  design  method  called  the  Procedurally 
Hierarchical  Issue-Based  Information  System  (PHIBIS)  method  at  the  University  of 
Colorado  at  Boulder.  PHIBIS  is  based  on  the  Issue-Serve  Systems  theory  of  design, 
which  views  the  design  process  as  an  information  seeking  process.  According  to  this 
theory,  the  act  of  designing  is  "an  attempt  to  devise  an  appropriate  plan  for  something  to  be 
built  or  implemented.”  Models  of  design  processes  are  thus  quasihierarchies  of  questions 
(issues)  that  arise  during  the  design  project  and  the  serve  relationships  that  are  developed 
among  the  issues.  Serve  relationships  describe  dependencies  among  the  issues. 

The  goal  of  the  PHIBIS  method  is  to  describe  and  improve  design  processes. 
Design  processes  are  described  by  means  of  representations  that  "account  for  the  full 
spectrum  of  graphic,  mathematical,  and  verbal  information  processes  which  occur  in 
design."  Information  processes  are  considered  paramount  during  design  activity  to  answer 
the  questions  that  may  arise. 

The  PHIBIS  method  addresses  improvements  to  design  processes  in  two  respects. 
First,  it  provides  the  means  to  document  design  issues  for  communication  among  project 
members.  This  mechanism  consists  of  knowledge  bases  for  structuring  individual  issues, 
positions,  arguments,  resolutions,  explanations,  and  comments  and  for  structuring  the 
serve  relationships  among  issues.  Second,  it  "uses  strategies  for  raising  important  design 
issues."  These  strategies  consist  of  procedures  for  systematically  structuring  the  design 
process. 

Theoretically,  the  PHIBIS  method  can  be  implemented  without  computers.  After 
analyzing  an  attempt  to  manually  implement  this  method  in  1979,  however,  the  author 
concluded  that  the  data  base  was  too  large  and  complex  to  manipulate  manually.  As  a 
result,  the  author  developed  software  (called  MICROPOLIS)  to  implement  the  PHIBIS 
method.  This  implementation  is  currently  under  further  development  and  testing  at 
Boulder.  Current  results  from  this  project  indicate  that  this  method 

•  "Becomes  a  vehicle  for  critical  discussion  among  the  various  parties  involved 
in  a  design  project.. 

•  Provides  a  vehicle  for  cumulative  growth  of  discourse  about  recurring  issues  in 
design... 

•  Provides  an  opportunity  for  researchers  to  see  into  and  study  design  processes 
and  information... 
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•  Provides  a  vehicle  for  dialog  between  research  and  practice... 

•  Allows  the  designer  to  lay  out  the  various  issues,  options,  and  arguments  of  a 
design  project  in  a  systematic  and  orderly  manner... 

•  Manages  the  complexity  and  controversy... 

•  Provides  a  useful  bookkeeping  system  for  tracking  the  current  state  of  thought 
and  discussion."  [Ref.  28] 

d.  Maher 

Maher  [Ref.  29]  is  developing  a  knowledge-based  approach  for  conceptual  design 
at  Carnegie  Mellon  University.  The  goal  of  this  research  is  "to  provide  a  domain 
independent  environment  for  developing  a  knowledge-based  system  for  preliminary  design 
synthesis"  that  is  sufficiently  expressive  for  representing  complex  systems  and  design 
expertise  and  experience. 

The  approach  under  development  incorporates  the  concepts  of  problem 
decomposition  and  constraint  satisfaction  to  model  design  as  a  search  process.  The  design 
space  (in  which  the  search  occurs)  consists  of  design  goals,  goal  solutions,  and 
relationships  among  goals.  This  knowledge  forms  the  basis  for  reasoning  about  design. 
Design  decisions  are  represented  in  the  form  of  knowledge  about  operators  that  act  on 
elements  of  the  design  space.  The  author  distinguishes  two  types  of  design  decisions- 
planning  knowledge  and  design  knowledge. 

Planning  knowledge  includes  information  on  which  design  goals  are 
relevant  to  the  current  situation.. .Planning  knowledge  is  incorporated  in  the 
design  process  so  that  the  decomposition  is  refined  as  the  solution  proceeds. 

The  rules  for  ordering  the  design  goals  can  consider  the  requirements  of  the 
given  design  problem  and  the  current  state  of  the  design  solution... 

"Design  knowledge  includes  information  on  which  design  solutions  are 
feasible  for  the  current  situation.  The  design  knowledge  is  used  to  guide  the 
selection  of  solutions  for  individual  goals. ..The  design  knowledge  is 
specified  by  the  design  in  the  form  of  alternative  solutions  for  each  goal  and 
constraints  on  combinations  of  solutions.  [Ref.  29] 

This  approach  is  being  implemented  by  the  authors  in  a  frame-based  system  called 
Engineering  Design  Synthesis  (EDESYN).  Design  knowledge  is  encoded  in  hierarchical 
frame  representations.  Both  planning  knowledge  and  design  knowledge  are  represented  in 
tree  structures,  where  paths  through  the  trees  represent  feasible  design  alternatives. 
Control  is  distinct  from  the  knowledge.  The  search  mechanisms  make  use  of  the  design 
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knowledge  represented  in  the  tree  structures  to  make  design  decisions.  Although  the 
implementation  is  not  complete  and  general  conclusions  have  not  been  drawn,  the 
following  aspects  of  this  research  are  significant; 

•  With  this  approach,  the  design  process  is  planned  dynamically  as  design 
proceeds. 

•  All  feasible  solutions  to  the  design  problem  are  generated,  not  just  the 
"optimal"  solution. 

•  The  approach  (both  the  knowledge  representation  scheme  and  the  control 
mechanism)  are  thoroughly  domain  independent. 

2 .  Strengths  and  Weaknesses  of  the  Approach 

a.  Strengths 

•  The  techniques  under  development  will  provide  the  means  to  formalize  what  is 
known  about  engineering  design  data  and  knowledge.  These  formalizations 
provide  the  basis  for  rigorous  analyses  of  engineering  design  processes. 

•  Using  knowledge  representation  techniques,  investigations  into  the  manner  in 
which  humans  use  knowledge  can  be  conducted.  Such  investigations  may  test 
theories  about  design  procedures,  what  design  decisions  are  made,  how  the 
decisions  are  made,  and  the  reasons  for  the  decisions. 

b.  Weaknesses 

•  "There  are  two  major  obstacles  to  integrating  expert  systems  into  the  design 
process.  The  first  problem  is  that  expert  systems  don't  yet  handle  geometric  or 
spatial  models  effectively.  Thus,  current  expert  systems  are  limited  in  their 
ability  to  assist  in  the  design  of  physical  forms. ..The  second  and  most 
pervasive  obstacle  is  that  design  does  not  have  a  complete  set  of  explicit  rules. 
Much  research  has  and  is  currently  being  performed  on  human  problem 
solving  and  the  design  process.  However,  even  when  rules  are  determined 
which  reflect  human  expertise,  much  of  the  information  in  the  knowledge  base 
of  a  typical  expert  system  is  either  imprecise,  incomplete,  or  not  totally  reliable. 
For  this  reason,  the  answer  to  a  question  or  the  advice  rendered  by  an  expert 
system  is  usually  qualified  with  a  certainty  factor..."  [Ref.  25] 

•  Very  little  is  known  concerning  the  most  effective  ways  to  elicit  knowledge 
from  human  experts.  To  be  truly  effective,  knowledge  elicitation  techniques 
should  capture  the  manner  in  which  an  expert  encodes  and  retrieves  knowledge 
of  a  specific  domain.  In  the  past,  however,  knowledge  extracted  from  human 


experts  has  been  unnaturally  manipulated  to  match  preconceived  structures  that 
•  typically  do  not  accurately  reflect  the  human  expertise. 

•  There  are  no  known  exhaustive  methods  for  testing  or  validating  knowledge 
representations  for  completeness  and  consistency. 


Knowledge  representation  technology  has  not  yet  evolved  to  a  state  that  can 
handle  all  of  the  peculiarities  of  design  engineering  data  and  knowledge.  Much 
basic  research  is  still  required. 
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III.  INDIVIDUAL  DESIGN  RESEARCH  STUDIES 


This  chapter  addresses  two  general  methods  for  stucying  the  design  of  a  product  by 
an  individual;  protocol  analysis  and  computational  experiments  of  problem-solving 
behavior. 

A.  PROTOCOL  ANALYSIS 

Protocol  analysis  is  an  empirical  method  for  studying  problem-solving  behavior 
exhibited  by  individuals  through  the  analysis  of  verbal  reports  (protocols).  The  method  is 
based  on  Newell  and  Simon's  information  processing  theory  (see  Section  II.D).  Protocol 
analysis  is  a  systematic  methodology  for  identifying  cognitive  processes  ("sequences  of 
internal  states  successively  transformed  by  a  series  of  operators"  [Ref.  30])  and  revealing 
the  nature  of  the  information  stored  and  accessed  by  human  beings  during  a  particular 
problem-solving  situation.  With  this  information,  the  manner  in  which  human  beings  solve 
problems  can  be  described. 

The  suppositions  that  this  method  is  based  on  include 

•  Thinking  aloud  does  not  alter  the  sequence  of  cognitive  processing 
significantly. 

•  Concurrent  verbal  and  retrospective  reports  provide  a  nearly  complete  record  of 
the  sequence  of  information  that  is  needed  during  task  performance. 

•  Verbally  reported  information  is  as  regular  and  valid  as  other  types  of  data. 
[Ref.  30] 

To  conduct  protocol  analysis  research,  a  lengthy  procedure  involving  preparation, 
data  collection,  and  extensive  data  analysis  is  typically  followed.  Preparation  involves 
designing  a  realistic  problem-solving  experiment,  identifying  subjects  who  have  an 
appropriate  level  of  experience  in  the  problem  domain  of  interest,  and  training  the  subjects. 
The  subjects  are  trained  to  verbalize  their  mental  processes  while  performing  design 
activities. 
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Data  collection  involves  recording  all  verbalizations  made  by  the  subjects  in  the 
course  of  solving  a  given  problem.  In  some  cases,  all  gestures  and  other  movements  (such 
as  eye  movements)  exhibited  by  the  subjects  and  all  products  (both  intermediate  and  final) 
are  filmed.  This  phase  of  the  experiment  produces  a  deluge  of  detailed  data  to  be 
interpreted  and  analyzed  by  the  researchers. 

Prior  to  data  analysis,  primitive  elements  that  may  constitute  a  general  problem¬ 
solving  strategy  are  hypothesized.  For  example,  one  set  of  elements  may  describe  the 
types  of  verbalizations  that  are  made  during  problem  solving.  Data  analysis  involves 
encoding  chunks  of  verbalizations  with  respect  to  the  predefined  elements.  Ideally,  the 
encoding  process  will  reveal  generalized  patterns,  in  terms  of  the  elements,  describing  the 
information  processing  strategy  used  to  solve  the  problem.  Encoding  can  be  achieved  only 
if  the  verbal  data  can  be  characterized  in  terms  of  the  predetermined  elements. 

1 .  Representative  Research  Studies 

Protocol  analysis  has  been  applied  rigorously  to  the  study  of  the  engineering  design 
process  only  recently,  although  many  early  design  studies  made  use  of  recorded  verbal  data 
to  substantiate  hypotheses  regarding  cognitive  aspects  of  the  design  decision  process.  A 
particularly  good  example  of  an  early  study  that  made  use  of  recorded  verbal  data  to  study 
selected  aspects  of  the  design  process  was  done  by  Malhotra,  Thomas,  Carroll,  and  Miller 
[Ref.  31].  The  research  methodology  developed  for  this  study  can  be  considered  a 
precursor  of  the  protocol  analysis  methodology.  Two  extremely  recent  studies  have 
rigorously  applied  protocol  analysis  to  test  the  validity  of  the  method  for  design  research 
purposes  [Ref.  26]  and  to  develop  non-trivial  models  of  the  mechanical  engineering  design 
process  [Ref.  32].  The  remainder  of  this  section  describes  the  results  from  these  studies. 

a.  Malhotra,  Thomas,  Carroll,  and  Miller 

The  focus  of  the  Malhotra,  Thomas,  Carroll,  and  Miller  study  [Ref.  3 1  ]  was  to 
"identify  and  characterize  important  design  processes... to  attain  an  understanding  of  the 
kinds  of  aids  that  can  assist  the  design  process."  Their  research  is  based  on  the  information 
processing  model  described  in  the  preceding  paragraphs.  Specifically,  the  authors  describe 
the  problem-solving  paradigm  as  follows: 

A  problem  state  is  said  to  exist  when  a  human  has  a  goal  but  no  immediate 

procedure  that  will  guarantee  attainment  of  the  goal.  The  goal  may  be  a 

satisfaction  to  be  achieved  or  a  dissatisfaction  to  be  alleviated.  Problem- 


solving  occurs  in  moving  from  a  problem  state  to  a  non-problem  state.  In 
problem-solving,  then,  a  person  begins  in  an  initial  state,  uses 
transformations  that  move  him  from  one  state  to  another  and  ends  in  a  final 
state.  Any  of  these  states  and  transformations  may  be  well-defined  or  ill- 
defined.  [Ref.  31] 

The  researchers  point  out,  however,  a  significant  difference  between  design  and 
general  problem  solving: 

In  the  case  of  design  problems,  the  person  is  generally  not  forced  to  start 
from  a  specific  initial  state,  and  although  there  may  be  constraints  on  what 
may  be  used,  the  transformations  are  usually  not  limited.  In  real  world 
design  situations  the  goals  are,  typically,  fuzzy  and  poorly  articulated  and 
cannot  be  mapped  directly  into  properties  of  the  design.  Thus,  the  exact 
configuration  of  the  final  state  is  not  prescribed.  A  part  of  the  design 
process  consists  of  formalizing  and  refining  the  design  goals  into  functional 
requirements  that  can  be  matched  by  properties  of  design.  [Ref.  31] 

• 

The  authors  analyzed  verbal  protocols  recorded  during  an  observational  study  of  a 
client-designer  dialogue  to  test  this  paradigm  in  the  context  of  conceptual  design.  The 
results  from  the  analysis  of  the  protocol  data  suggested  a  model  that  illustrates  the  cyclical 
nature  of  early  client-designer  interactions.  According  to  the  results,  decisions  made  during 
this  conceptual  phase  occur  in  a  "series  of  cycles;  each  cycle  consisting  of  a  regular 
succession  of  states.  A  state  is  defined  as  a  portion  of  the  dialogue  that  is  oriented  towards 
a  single  purpose."  The  six  states  identified  are  listed  in  Table  III-l.  Thus,  each  cycle 
"consists  of  introducing  some  requirements,  discussing  them,  outlining  a  solution,  and 
elaborating  and  exploring  it." 

Table  111-1.  States  in  the  Client-Designer  Dialogue 


State 

Main  Speaker 

1 .  Goal  Statement 

Client 

2.  Goal  Elaboration 

Client  and  Designer 

3.  Sub  Solution  Outline 

Designer 

4.  Sub  Solution  Elaboration 

Designer 

5.  Sub  Solution  Explication 

Client  and  Designer 

6.  Agreement  on  Sub  Solution 

Client 

From  the  client-designer  dialogue  experiment  and  two  other  related  experiments,  the 
researchers  developed  a  general  model  of  the  design  process.  This  model  decomposes  the 
design  process  into  three  fundamental  underlying  processes:  goal  elaboration,  design 
generation,  and  design  evaluation.  Goal  elaboration  corresponds  to  states  1  and  2.  Design 
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generation  corresponds  to  states  3, 4,  and  5.  Design  evaluation  corresponds  to  states  5  and 
6.  These  processes  are  interacting  processes  and  may  occur  concurrently. 

The  Malhotra  study  is  an  early  attempt  to  propose  a  "framework  within  which 
specific  hypotheses  can  be  empirically  tested"  [Ref.  31].  In  particular,  this  framework  can 
be  used  to  investigate  decision  support  research  issues  such  as  the  following: 

To  what  extent  do  traditional,  fact-oriented  information  retrieval  systems 
provide  substantive  support  to  designers? 

What  are  the  advantages  and  disadvantages  of  rapid  prototyping  in  terms  of 
quickly  seeing  alternative  solutions,  prematurely  focusing  on  initial  ideas, 
etc.? 

To  what  extent  are  on-line  expert  systems  likely  to  be  of  value  for  different 
types  of  design  problem  and  phases  of  design? 

In  what  ways  might  current  computer-aided  design,  computer-aided 
manufacturing,  and  computer-aided  engineering  (CAD/CAM/CAE)  systems 
be  enhanced,  within  current  technology  constraints,  to  provide  facts, 
fantasies,  and  feedback? 

Is  it  likely  that  designers  will  be  confused  or  dissatisfied  by  a  support 
system  whose  role  is  adapted  to  the  nature  and  state  of  their  design  problem 
solving? 

What  other  roles  can  be  imagined  for  a  design  support  system,  and  when 
might  these  roles  be  invoked?  [Ref.  3] 

b.  Eckersley 

Eckersley  [Ref.  33]  used  protocol  analysis  in  a  study  primarily  to  determine 
whether  or  not  it  is  "an  effective  method  for  the  controlled  observation  and  experimental 
analysis  of  design  problem-solving  behavior."  To  validate  the  efficacy  of  this  method, 
Eckersley  formulated  a  rather  general  experiment  hypothesis  that  designers  vary 
significantly  in  the  nature  and  amount  of  information  processed  during  problem  solving. 

This  hypothesis  was  tested  by  analyzing  verbal  protocols  with  respect  to  predefined 
verbalization  categories.  The  verbalization  categories,  or  types  of  statements,  are  primitive 
elements  that  may  constitute  a  problem-solving  strategy.  The  types  of  statements  that  the 
researchers  expected  to  find  in  tlie  protocols  included 

•  Literal  copy  statements 

•  Paraphrased  copy  statements 

•  Inferences 
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•  Intentions/plans 

•  Move  statements 

•  Search  statements 

•  Specific  assessments 

•  General  assessments. 

With  these  verbalization  categories,  the  authors  attempted  to  model  problem-solving 
behavior  as  patterns  in  the  occurrences  of  each  type  of  statement  during  the  design  process. 
These  verbalization  categories  correspond  to  the  types  of  information  processed  during 
problem  solving.  An  analysis  of  the  protocols,  however,  revealed  no  general  patterns 
across  all  subjects.  The  author  concluded  that  the  lack  of  consistent  verbalization  patterns 
supports  the  working  hypothesis.  The  analysis  also  supports  an  informal  hypothesis  that 
stages  in  problem  solving  would  be  evident  in  the  analysis  of  protocol  data. 

The  author  concludes  that  the  study  was  successful  in  testing  protocol  analysis  and 
suggests  "much  work  remains  to  be  done  in  order  to  establish  protocol  analysis  as  a  valid 
design  research  tool.  The  potential  for  experimentally  testing  research  hypotheses  is  great, 
provided  larger  subject-samples  are  utilized." 

c.  Ullman,  Dietterich,  and  Stauffer 

Ullman,  Dietterich,  and  Stauffer  [Ref.  32]  recently  completed  a  two-year  study  of 
the  mechanical  design  process  using  the  protocol  analysis  methodology  to  examine  how 
experienced  mechanical  designers  solve  real  design  problems.  This  method  provided  the 
researchers  with  a  systematic  procedure  for  generating  and  analyzing  empirical  data. 

The  focus  of  the  analysis  of  copious  amounts  of  protocol  data  was  to  identify 
relationships  between  form,  function,  and  design  decisi  is.  This  led  to  the  development  of 
a  model  of  the  design  process  referred  to  as  the  Task-Episode  Accumulation  Model 
(TEAM).  TEAM  "explains  many  aspects  of  the  design  process  and  it  provides  significant 
insights  into  the  the  way  designs  are  developed." 

The  concepts  underlying  TEAM  correspond  closely  with  the  Newell  and  Simon 
information  processing  theory  [see  Section  II.D].  TEAM  describes  a  problem  space  in 
which  design  states  are  transformed  by  design  operators. 

The  design  state  contains  all  information  about  the  evolving  design 

including  problem  specifications,  additional  constraints  introduced  by  the 
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designer,  proposed  designs,  drawings,  calculations,  assembly  plans,  and  so 
on. 

Design  operators  are  primitive  information  processes  that  modify  the  design 
state  by  performing  calculations,  creating  new  proposed  designs,  evaluating 
proposed  designs,  and  making  decisions  to  accept  or  reject  proposed 
designs.  TEAM  contains  ten  operators:  select,  create,  simulate,  calculate, 
compare,  accept,  reject,  suspend,  patch,  and  refine.  [Ref.  32] 

To  experiment  with  the  concept  of  a  design  state,  the  authors  developed  a 
comprehensive  representation  scheme  for  the  data  used  during  design  and  identified  data 
representation  requirements.  A  frame-based  approach  to  data  representation  satisfies  these 
requirements. 

In  the  TEAM  model,  operators  are  grouped  into  meaningful  sequences,  called 
episodes,  each  of  which  addresses  a  low-level  goal.  Heuristic  rules  guide  the  decision¬ 
making  process  with  respect  to  the  use  of  operators  within  an  episode.  An  examination  of 
the  episodes  reveals  design  problem-solving  strategies  used  by  the  subjects.  These 
strategies  describe  the  manner  in  which  a  present  state  of  the  design  was  transformed  to 
meet  design  goals.  Strategies  exhibited  by  the  subjects  include 

•  "Designers  often  pursued  a  single  conceptual  design  both  at  the  level  of  the 
overall  design  problem  and  at  the  level  of  individual  subproblems. 

•  Designers  used  notes  and  drawings  for  understanding  and  analyzing  the 
problem,  not  just  to  record  final  design  decisions. 

•  Designers  progressed  from  systematic  to  opportunistic  behavior  as  the  design 
evolved. 

•  Designers  did  not  always  conduct  balanced  development  but  sometimes 
pursued  a  problem  in  a  depth-first  manner. 

•  Designers  sometimes  repeated  previous  efforts  in  solving  the  problem. 

•  Form  and  function  are  interrelated  in  the  design  process. 

•  Designers  based  many  decisions  on  qualitative  rather  than  quantitative 
reasoning  during  all  phases  of  design. 

•  Knowledge  controls  the  design  processes. 

•  Designers  made  simulations  as  an  aid  to  understanding  problems  and 
evaluating  solutions. 

•  Designers  usually  found  satisfactory  solutions  rather  than  optimal  solutions." 
[Ref  32] 
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The  importance  of  the  TEAM  model  is  that  in  addition  to  explaining  "the  design 
process  to  the  level  of  individual  utterances  in  the  protocol  data,"  [Ref.  32]  it  provides 
many  new  insights  into  the  design  problem-solving  behavior  of  individual  designers.  The 
authors  suggest  a  number  of  implications  these  results  have  on  the  development  of  design 
aids.  For  instance: 

•  "Existing  CAD  tools  are  addressing  primarily  the  detail  design  tasks,  because 
CAD  tools  are  only  capable  of  representing  the  geometric  design  state  when  it 
is  refmed  to  a  concrete  or  near-concrete  level."  [Ref.  32] 

•  Users  of  CAD  tools  would  benefit  from  constraint  management  assistance. 

•  "Existing  artificial  intelligence  models  of  design  are  still  rather  simplistic  and 
inflexible."  [Ref.  32] 

•  TEAM  will  be  refined  and  augmented  with  results  from  a  recently  begun  three-year 
project  to  continue  the  examination  of  the  design  process  and  "to  design  and  implement  a 
design  history  tool." 

2 .  Strengths  and  Weaknesses  of  the  Approach 

The  following  strength  statements  made  by  a  variety  of  researchers  who  have 
studied  or  made  use  of  the  protocol  analysis  methodology  provide  support  for  the  potential 
of  protocol  analysis  for  studying  the  individual  human  problem-solving  process.  These 
statements  also  suggest  that  this  methodology  is  still  under  development;  results  suggested 
by  protocol  analysis  studies  must  be  used  with  caution. 

a.  Strengths 

•  "Since  thought  is  not  a  directly  observable  activity,  introspective  and 
retrospective  accounts  of  human  thought  have  rightly  been  regarded  as 
unreliable  data  for  scientific  enquiry.  However,.. .under  controlled  conditions, 
’ndividuals...can  reveal  a  remarkably  accurate  picture  of  their  cognitive 
processes  while  engaged  in  problem  solving. ...Protocol  analysis  offers  the 
community  of  design  scientists... a  potentially  effective  method  for  the 
controlled  observation  and  experimental  analysis  of  design  problem-solving 
behavior."  [Ref.  33] 

•  "The  power  of  this  approach  lies  in  its  richness  of  data.  Fragmentary  glimpses 
of  complex  processes  provide  more  information  about  the  processes  than 
simply  the  outcome."  [Ref  34] 
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The  resulting  model  can  be  used  to  guide  research  and  as  a  baseline  for  refined 
models.  The  method  can  be  used  to  test  models  (i.e.,  which  parts  are  correct, 
which  need  to  be  refined  or  replaced).  [Ref.  34] 

"The  potential  for  experimentally  testing  research  hypotheses  is  great,  provided 
larger  subject-samples  are  utilized."  [Ref.  33] 

Introspection  is  useful  for  discovery  purposes.  [Ref.  30] 

This  method  is  useful  for  eliciting  much  valuable  information  about  all  aspects 
of  design  activity  that  designers  have  difficulty  expressing. 

"Educationally,  protocol  analysis  could  be  of  potential  use  as  a  diagnostic  tool, 
or  as  an  instrument  to  show  students  the  'form'  of  their  problem-solving,  of 
which  they  may  be  singularly  unconscious."  [Ref.  33] 


b.  Weaknesses 

•  The  suitability  of  the  subject's  verbalizations  as  scientific  data  is  debatable. 
[Ref.  30] 

•  "The  process  of  verbalization  disrupts  thought  processes."  [Ref.  21]  "The 
requirements  to  'think  out  loud'  tend  to  slow  subjects  down,  [although] 
research  has  shown  that  this  does  not  affect  the  order  or  content  of  their 
problem  solving  steps."  [Ref.  32] 

•  "Verbal  protocols... cannot  record  what  happens  during  incubation,  when  the 
subject  'mulls  over'  the  problem  for  a  few  days  and  thinks  up  ideas." 
[Ref.  32] 

•  Encoding  behavior  into  data  is  subjective  and  prone  to  bias.  Different 
interpreters  will  arrive  at  an  interpretation  that  is  consistent  with  the 
investigator's  particular  theoretical  orientation.  [Ref.  30] 

•  The  theoretical  presuppositions  are  embedded  in  the  encoding  process.  [Ref. 
30] 

•  The  investigators  must  be  concerned  with  methodological  questions  about  data 
collection.  [Ref.  30] 

•  Protocols  will  always  be  incomplete.  Processes  occur  that  the  subject  cannot 
or  does  not  report.  Furthermore,  the  subject  can  only  report  a  linear  sequence 
of  thoughts,  one  thought  at  a  time.  Unstated  thoughts  regarding  form, 
function,  and  process  must  be  inferred  by  the  investigators. 

•  It  is  difficult  to  study  the  entire  design  process  in  one  session.  The  size  of  the 
design  problem  requires  several  sessions.  After  two  to  three  hours  fatigue  sets 
in.  [Ref.  32]. 
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•  "Real-time  protocol  analysis  has  limitations  based  on  the  volume  of 
information  which  must  be  collected  and  sorted.  The  integrated  visual,  spatial, 
and  verbal  nature  of  the  knowledge  and  decisions  made  by  a  group  of 
individuals  is  often  inadequately  represented  by  this  mode  of  data  collection." 
[Ref.  21] 

•  It  is  best  suited  to  studying  subjects  solving  problems  by  themselves. 
Engineers,  however,  often  design  in  teams  or  at  least  pass  their  problems 
around  for  others  to  review.  [Ullman,  et  al.]  could  not  think  of  any  practical 
way  of  accommodating  this  aspect  of  design  in  the  present  study."  [Ref.  32] 

B  .  COMPUTATIONAL  EXPERIMENTS  OF  PROBLEM-SOLVING 
BEHAVIOR 

Many  design  research  programs  focus  on  investigating  theories  and  hypotheses 
regarding  human  design  problem  solving  by  conducting  empirical  experiments.  These 
experiments,  both  laboratory  and  computational,  are  purposely  designed  and  controlled  to 
generate  empirical  evidence  of  cognitive  design  decision  activity  at  the  level  of  the 
individual.  By  analyzing  the  resulting  data  and  identifying  general  patterns,  the  researchers 
are  able  to  substantiate,  refine,  or  refute  research  hypotheses. 

The  resulting  data  are  reduced  and  analyzed  statistically.  Analysis  results  are  used 
to  test  the  original  hypothesized  models  of  design  activity  and  set  forth  new  ideas  about 
design  problem-solving  behavior. 

Computational  experiments  are  an  alternative  approach  to  traditional  laboratory 
methods  for  testing  models  of  the  design  process.  The  purpose  of  computational 
experiments  is  similar  in  intent  to  laboratory  experiments.  After  conceptualizing  a  model  of 
a  specific  aspect  of  cognitive  problem-solving  activity,  the  model  is  implemented  in 
software.  The  resulting  computational  program  is  evaluated  to  validate  the  original 
hypotheses  and  identify  weaknesses  of  the  model.  The  value  of  the  computer  as  a  research 
tool  used  in  this  context  is  described  by  Gasparski,  et  al.  [Ref.  35]: 

Development  of  computer  science  has  greatly  contributed  to  the 

development  of  research  on  the  processes  of  solving  design  problems. 

Analysis  of  the  very  design  problems  became  an  alternative  to  drawing 

conclusions  from  observation  of  patterns  of  the  design  process. 

Furthermore,  the  evolution  of  artificial  intelligence  techniques  that  aim  to  render 
non-deterministic  processes  computable,  has  contributed  significantly  to  the  arsenal  of 
viable  techniques  for  testing  models  of  the  design  problem-solving  process. 
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An  important  distinction  exists  between  programs  that  design  and  programs  to  test 
theories  about  how  people  design  artifacts.  The  former  are  considered  computational  aids 
for  the  design  process.  They  typically  address  specific  segments  of  the  detailed  phases  of 
design.  The  tasks  that  they  perform  involve  routine  or  mechanistic  design  decisions  and 
are  very  discipline  specific.  The  latter,  however,  are  computational  models  of  design 
processes.  The  usual  focus  of  these  programs  is  the  conceptual  phase  of  design, 
characterized  by  synthesis  tasks  that  require  innovative  solutions  to  complex  or  ill-defined 
design  decisions. 

The  next  section  describes  three  experiments  to  test  theories  of  human  design 
problem-solving  activity  by  implementing  computational  models  of  specific  aspects  of  the 
design  decision  process. 

1 .  Representative  Research  Studies 

The  design  research  literature  contains  a  number  of  interesting  reports  of 
computational  experiments  aimed  at  gathering  empirical  evidence  to  refine  models  of 
various  aspects  of  the  design  process  at  the  level  of  individual  activity.  Results  from  these 
experiments  have  contributed  significantly  toward  a  deeper  understanding  of  cognitive 
reasoning  processes  underlying  design  problem-solving  behavior. 

The  three  research  experiments  described  in  this  section  form  a  representational 
sampling  of  computational  experiments  that  have  been  and  are  being  conducted.  The 
design  decision  process  models  that  have  resulted  from  these  three  projects  provide 
evidence  of 

•  The  manner  in  which  designers  use  knowledge  and  skills  to  produce  a  novel 
design 

•  The  nature  of  operational  knowledge  and  goal  specifications  necessary  to  make 
correct  design  decisions 

•  The  role  of  episodic  memory  and  analogical  reasoning 

•  Stages  of  reasoning  used  during  design 

•  The  manner  in  which  design  knowledge  is  formed  and  design  experience  is 
acquired  and  formalized 

•  Aspects  of  the  design  process  that  should  be  subjected  to  analysis 


•  Requirements  for  generic  processes  of  invention 

•  The  role  of  rules  of  invention  and  creativity. 

a.  Dyer  and  Flowers 

Dyer  and  Flowers  [Ref.  36]  are  exploring  applications  of  artificial  intelligence  (AI) 
to  design  at  the  University  of  California  at  Los  Angeles.  They  are  particularly  interested  in 
exploring  the  cognitive  skills  of  mechanical  design  engineers.  With  an  understanding  of 
human  problem-solving  strategies,  they  hope  to  create  AI  systems  that  will  aid  the  design 
process.  Their  research  reported  in  this  study  addresses  the  question  "of  whether  and  to 
what  extent,  processes  of  creative  thought  can  be  encoded  into  a  computer  so  that  the 
computer  can  be  said  to  have  invented  something  on  its  own."  [Ref.  36] 

The  research  is  based  on  Newell  and  Simon's  information  processing  model  of 
problem  solving  [see  Section  II.D],  The  inventive  aspect  of  the  mechanical  design  process 

is  viewed  as  a  search  through  a  large  space  of  states  representing  objects  or 
ideas.  Movement  from  one  state  to  another  is  accomplished  by  the 
application  of  an  operator.  In  our  case,  the  state  space  consists  of  object 
desi^s  and  the  operators  are  'creativity  rules'  or  rule  schemas  which 
modify  or  combine  existing  objects  to  yield  new  designs.  [Ref.  36] 

The  rule  schemas  direct  the  inventive  design  decision  process.  The  purpose  of  the 
rules  schemas  is  to  embody  the  manner  in  which  a  designer  uses  knowledge  and  skills  to 
produce  a  novel  design. 

To  test  the  feasibility  of  applying  AI  to  design  with  this  paradigm,  the  authors 
attempted  to  automate  the  invention  of  "a  novel  version  of  a  familiar  device-i.e.,  the  door." 
[Ref.  36]  Heuristic  rules  of  creativity  were  developed  using  informal  introspective 
protocols  (the  authors  documented  what  they  believed  they  were  doing  cognitively  while 
inventing  a  novel  device).  These  rules  were  implemented  in  an  expert  system  (the 
Intelligent  Design  Invention  System). 

The  following  conclusions  were  drawn  from  the  results  of  the  computational 
experiment: 

•  Problem  solving  is  central  to  invention. 

•  Operational  knowledge  (i.e.,  common  sense  knowledge)  is  necessary  to  make 
correct  decisions. 
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•  Functional  and  goal  specifications  are  necessary  to  make  correct  design 
decisions. 

•  The  ability  to  distinguish  novel  solutions  from  routine  solutions  is  necessary  to 
make  correct  design  decisions. 

•  Episodic  memory  and  analogical  reasoning  is  crucial  for  creative  problem¬ 
solving  (for  instance  it  is  important  to  be  able  to  "take  an  object  from  a 
completely  different  domain  and  apply  it  to  the  current  task  domain,  altering  it 
as  needed").  [Ref.  36] 

The  design  decision  process  model  that  resulted  from  this  research  is  in  the  form  of 
a  collection  of  expert  system  rule  schemas.  These  rule  schemas  provide  the  expert  system 
with  necessary  information  and  guidelines  to  make  design  decisions  during  attempts  to 
invent  some  novel  device. 

b.  Findler 

Findler  [Ref.  5],  at  the  State  University  of  New  York  (SUNY)  at  Buffalo,  is 
studying  the  role  of  analogical  reasoning  in  problem  solving  by  computers  with  attention 
directed  towards  design  processes.  He  views  design  as  an  ill-structured  problem  that  is 
difficult  to  solve  by  any  prescribed  algorithm.  To  study  the  role  of  analogical  reasoning  in 
the  design  process  certain  assumptions  about  the  design  process  are  made.  Specifically: 

•  Each  design  problem  is  describable  as  an  (ordered)  collection  of  certain, 
possibly  overlapping,  fundamental  features. 

•  Solutions  are  associated  with  respective  problems  in  a  well-defined, 
deterministic  manner... .The  features  of  the  problems  are  identifiable  and  are 
strongly  enough  correlated  with  the  solutions  so  that  the  latter  can  be  directly 
derived  from  the  former. 

•  In  the  task  domains  of  interest  to  us,  similar  problems  have  similar  solutions. 

•  When  two  problems  have  similar  solutions,  the  features  present  in  one  problem 
but  not  in  the  other  are  likely  to  be  of  lesser  importance.  In  turn,  features 
shared  by  problems  which  have  similar  solutions  are  likely  to  be  important. 
[Ref.  5] 

Findler  conducted  several  computational  experiments  to  identify  "what  information 
should  be  extracted  from  raw  experience  consisting  of  descriptions  of  problems  and 
solutions,  and  how  this  information  can  then  be  used  in  determining  solutions  to  new 


problems."  [Ref.  5]  These  experiments  consisted  of  implementing  two  different  yet 
mutually  supportive  models  of  analogical  reasoning  mechanisms; 


Contributive  model:  A  solution  to  the  problem  is  constructed  incrementally 
by  (1)  generating  transformation  functions  between  features  of  previous 
problems  and  solutions,  and(  2)  applying  the  transformation  functions  to  the 
current  problem  to  identify  a  solution.  [Figure  Ill-l(a)] 


Hierarchical  model:  The  hierarchical  relationships  between  problem  features 
are  accounted  for  in  this  model  by  (1)  finding  transformation  functions 
between  features  of  a  relevant  previous  problem  and  features  of  the  current 
problem,  and  (2)  applying  those  transformation  functions  to  the  solution  of 
the  most  relevant  previous  problem  to  obtain  a  solution  to  the  current 
problem.  [Figure  III- 1(b)] 


Figure  III-1.  Findler's  Models  of  Analogical  Reasoning 


These  experiments  resulted  in  two  general  conclusions: 

•  The  processes  of  analogical  reasoning  are  a  useful  and  general  component  of 
design  problem  solving. 
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•  The  computational  procedures  implemented  in  this  study  effectively  transform 
raw  experience  in  problem  solving  into  a  viable  knowledge  base  to  support 
design  decisionmaking. 

This  research  is  particularly  interesting  because  it  attempts  to  describe  analogical 
reasoning  mechanisms  that  collect  and  process  experiential  information  that  designers  use 
to  make  design  decisions.  It  pro /ides  models  of  cognitive  strategies  that  guide  the  search 
for  solutions  during  design.  These  models  form  a  framework  for  exploring  both  how 
experiential  knowledge  is  used  during  design  and  the  extent  to  which  learning  is  an  integral 
component  of  the  design  process. 

c.  Green  and  Bro>vn 

As  part  of  ongoing  research  into  routine  design.  Green  and  Brown  [Ref.  37]  at 
Worcester  Polytechnic  Institute  are  investigating  "knowledge  and  problem-solving  behavior 
used  during  routine  design."  The  background  for  this  project  is  explained  by  the  authors: 

Brown  and  Chandrasekaran  have  proposed  that  knowledge  about  routine 
design  is  distributed  throughout  a  tightly  organized,  hierarchical  structure. 
Problem-solving  activity  occurs  as  control  passes  among  the  pieces  of 
knowledge  in  this  hierarchy.  This  hierarchy  represents  a  very  "compiled" 
form  of  the  knowledge  as  all  the  alternatives  are  captured  in  the  structure  by 
alternative  design  plans,  or  by  simple  choices  in  design  knowledge. 

This  study  focuses  on  strategies  for  "compiling"  or  generating  a  specific  type  of 
design  knowledge,  constraint  knowledge.  The  specific  question  being  addressed  is  how  a 
mechanical  design  engineer  can  determine  the  content  of  a  constraint  with  respect  to  how' 
well  previously  designed  subcomponents  fit  together.  The  generation  of  constraint 
knowledge  of  this  type  requires  spatial  reasoning. 

The  authors  developed  a  four-stage,  sequential  yet  continuous,  model  of  the  spatial 
reasoning  process  to  explore  the  manner  in  which  constraint  knowledge  is  formed.  During 
each  stage,  a  unique  set  of  decisions  must  be  made  about  how  subcomponents  may  fit 
together.  The  stages  and  examples  of  corresponding  decisions  are  shown  in  Table  111-2. 

The  last  two  stages  of  this  model  have  been  implemented  and  tested  against  a 
problem  space  representing  a  very  simplistic  class  of  objects.  Early  results  from  this 
implementation  confirm  the  authors'  suspicion  that  the  spatial  reasoning  process  is  quite 
complex.  Limitations  in  the  current  implementation  suggest  necessary  extensions  to  the 
model  to  "cover  more  complex  and  realistic  objects."  As  a  result  of  this  implementation. 


Table  III-2.  Stages  in  the  Spatial  Reasoning  Process 


stage 

Decisions 

Grouping 

Which  features  of  the  object  are  extraneous  to  the  fit  problem  and  which  are  important? 

How  should  relevant  features  be  clustered? 

Orientation 

Which  surfaces  of  the  object  are  likely  candidates  for  matching? 

Should  the  object  be  rotated? 

Should  the  surface  be  rotated? 

Matching 

What  information  about  the  features  in  each  cluster  should  be  retrieved? 

Which  clusters  of  features  on  one  object  match  clusters  of  features  on  the  other'^ 

Confirmation 

Are  the  paired  features  compatible?  Was  a  fit  successful? 

the  authors  have  identified  several  important  types  of  knowledge  and  are  confident  that  the 
proposed  model  of  the  spatial  reasoning  process  is  tenable. 


2  .  Strengths  and  Weaknesses  of  the  Approach 

a.  Strengths 

•  Computational  experiments  can  result  in  a  wealth  of  raw  data  that  serves  as 
direct  evidence  to  support  models  of  design  decision  processes  at  the  level  of 
individual  activity. 

•  The  resultant  design  models  attempt  to  describe  how  people  actually  design  as 
opposed  to  describing  how  people  should  design. 

•  In  addition  to  attempting  to  describe  individual  design  decision  situations,  the 
resultant  models  also  attempt  to  provide  an  explanation  of  the  manner  in  which 
design  is  accomplished. 


b .  Weaknesses 

•  Theoretical  principles  underlying  a  general  science  of  design  have  yet  to  be 
postulated  and  accepted  by  the  engineering  design  research  community  at  large. 
Thus,  empirical  experiments  of  design  problem  solving,  while  undoubtedly 
contributing  to  the  development  of  design  principles,  are  not  based  on  a 
universal  framework  of  understanding. 

•  Computational  experiments  do  not  test  or  consider  all  types  of  problems 
engineers  may  expect  to  encounter  during  engineering  design. 
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•  The  experiments  examine  the  design  process  from  a  single  perspective. 

•  Reasoning  strategies  under  investigation  are  fixed  in  advance.  [Ref.  15] 


•  The  experimental  hypotheses  under  investigation  are  not  tested  in  a  real  world 
engineering  setting.  The  ways  in  which  a  subject  solves  a  design  problem  in  a 
laboratory,  or  a  machine  is  programmed  to  solve  a  design  problem,  are  not 
necessarily  related  to  the  manner  in  which  engineering  design  actually 
precedes. 
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IV.  OBSERVATIONS 


A.  DIVERSE  MOTIVATIONS 

A  diverse  number  of  motivations  drive  the  study  of  the  design  process  from  both 
theoretical  and  practical  points  of  view.  For  example,  there  are  pockets  of  investigators 
throughout  the  world  seeking  to 

•  Establish  a  science  of  design 

•  Examine  underlying  mental  processes  and  cognitive  strategies  followed  during 
a  problem-solving  activity 

•  Understand  the  industrial  and  economic  implications  of  engineering  design 
processes 

•  Develop  techniques  that  could  help  designers  in  certain  stages  of  their  work, 
such  as  brainstorming,  synectics,!  value  analysis,  morphology,  and  lateral 
thinking 

•  Develop  "systematic  procedures  to  guide  the  design  process"  (such  as  the 
German  VDI-Richtlinien)  [Ref.  4] 

•  Capture  design  experience  by  means  of  expert  systems,  etc.  [Ref.  4] 

•  Develop  improved  methods  for  teaching  design. 

The  abstracts  of  these  surveys  are  included  in  the  annotated  bibliography.  Anyone 
interested  in  engineering  design  research  should  examine  these  papers  in  detail  before 
initiating  related  work. 
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Synectics  is  the  study  of  creative  processes,  especially  as  applied  to  the  solution  of  problems  by  a 
group  of  diverse  individuals.  Synectics  comes  from  the  Greek  word  meaning  the  joining  together  of 
different  and  apparendy  irrelevant  elements. 


B .  COMMON  THEMES 

Many  common  themes  underlie  the  research  reported.  These  commonalities  include 

•  The  engineering  design  problem  is  an  "ill-structured"  problem,"  and  thus 
difficult  to  solve  by  any  set  algorithm."  [Ref.  5] 

•  Design  problems  are  hierarchical  and  can  be  decomposed  into  subproblems. 

•  Design  is  both  a  creative  and  analytical  process. 

•  Design  is  a  dynamic  and  iterative  process. 

•  Design  problem-solving  behavior  involves  analogical  reasoning. 

•  "The  nature  of  designing  requires  an  interdisciplinary  approach."  [Ref.  6] 

•  Design  is  both  a  technical  and  behavioral  process;  it  cannot  be  studied  as 
though  it  were  one  and  not  the  other.  [Ref.  7] 

•  Although  a  number  of  attempts  have  been  made  to  define  engineering  design, 
none  has  been  completely  successful.  [Ref.  8] 

•  Designers  do  not  attempt  to  find  optimum  solutions,  rather  they  attempt  to  find 
solutions  that  work  ("satisfice"). 

•  "Many  researchers  opt  for  a  systems  approach."  [Ref  9] 

•  Design  decisions  are  usually  not  independent. 

•  There  is  no  interdisciplinary  "language  of  design."  [Ref  6] 

•  There  are  few,  if  any,  automated  aids  for  conceptual  design.  Most  design  aids 
are  applicable  during  detail  design. 

C.  COMMON  DEFINITIONS 

Most  of  the  definitions  of  the  engineering  design  process  suggested  by  individual 
researchers  closely  match  those  of  the  Accreditation  Board  of  Engineering  and  Technology 
(ABET).  Engineering  design  is  described  by  ABET  as 

the  process  of  devising  a  system,  component,  or  process  to  meet  desired 
needs.  It  is  a  decision-making  process  (often  iterative),  in  which  the  basic 
sciences,  mathematics,  and  engineering  sciences  are  applied  to  convert 
resources  optimally  to  meet  a  stated  objective.  Among  the  fundamental 
elements  of  the  design  process  are  the  establishment  of  objectives  and 
criteria,  synthesis,  analysis,  construction,  testing,  and  evaluation.  [Ref  8] 


D.  SUMMARY 


This  report  has  reviewed  six  approaches  that  have  been  used  by  the  engineering 
design  research  community  to  examine  the  design  decision  process.  These  approaches 
yield  valuable  information  about  design  processes.  The  approaches,  however,  result  in 
significantly  different  types  of  design  decision  process  information.  Table  IV-1  is  a 
summary  of  the  types  of  design  decision  process  information  that  is  typically  produced. 

Table  IV-1.  Design  Decision  Process  Information 


Approach 

Design  Decision  Process  Modei 

Systems  Analysis 

Hierarchical,  systematic  diagrams  of  the  activities,  information  flows,  and 
influences  characterizing  a  typical  engineering  design  process 

Participant  Observation 

Quantitative  (graphs,  etc.)  and  qualitative  (narratives,  etc.)  descriptions  of 
the  activities  of  a  particular  design  project 

Retrospective  Analysis 

High-level  frameworks  describing  the  major  drivers  and  decision  points  of  past 
engineering  projects 

Knowledge 

Representation 

Formal  representations  of  design  decision  structures 

Protocol  Analysis 

High-level  frameworks  that  describe  the  basic  components  of  cognitive 
activity  associated  with  design  problem  solving 

Computational 

Experiments 

Executable  models  of  problem-solving  behavior 

The  research  results  are  all  considered  models  of  the  design  decision  process. 
Although  the  natures  of  the  models  listed  in  Table  IV-1  are  quite  distinct,  each 
representation  serves  as  a  tool  for  considering  the  design  process  problem.  Because  none 
of  the  models  considers  all  aspects  of  the  engineering  design  decision  process,  a 
combination  of  approaches  is  required  to  study  design  processes  in  their  entirety.  As  this 
paper  illustrates,  the  engineering  design  research  community  is  exploring  the  application  of 
different  types  of  methodologies  to  advance  its  understanding  of  the  design  decision 
process.  Motivation  for  this  work  is  well  stated  by  Simon: 

There  are  no  more  promising  or  important  targets  for  basic  scientific 
research  than  understanding  how  human  minds,  with  and  without  the  help 
of  computers,  solve  problems  and  make  decisions  effectively,  and 
improving  our  problem-solving  and  decision-making  capabilities.  [Ref.  10] 
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ANNOTATED  BIBLIOGRAPHY 


This  bibliography  contains  three  sections.  Section  A  lists  surveys  of  the 
engineering  design  research  literature.  Section  B  lists  articles  that  describe  engineering 
design  research  projects.  These  projects  are  organized  in  the  following  categories: 

Methods  to  model  organizational  design  decision  processes 

•  Systems  analysis 

•  Participant  observation 

•  Retrospective  analysis 

•  Expert  system  knowledge  representation. 

Methods  to  model  individual  design  decision  processes 

•  Protocol  analysis 

•  Laboratory  and  computational  experiments. 

Section  C  lists  books  and  articles  that  directly  relate  to  or  contribute  to  the 
understanding  of  the  research  studies  described  in  Section  B. 

A.  REVIEWS  OF  DESIGN  RESEARCH 

Eder,  Ernst.,  "Design  Science— A  Survey  of  Some  Approaches,"  Proceedings  of  the  1987 
ASEE  Annual  Conference. 

ABSTRACT:  "The  conditions  under  which  a  body  of  knowledge  can  be 
labelled  a  science  are  surveyed.  It  is  shown  that  if  engineering  design  can 
be  studied  in  its  widest  context  a  'design  science'  can  be  generated.  This 
conclusion  is  based  on  the  definition  as  follows: 

Engineering  design  is  a  process  performed  by  humans  aided  by  technical 
means  through  which  information  in  the  form  of  requirements  is  converted 
into  information  in  the  form  of  descriptions  of  technical  systems,  such  that 
this  technical  system  meets  the  requirements  of  mankind. 


In  order  to  establish  a  scientific  theory  of  design,  the  following  aspects  need 
to  be  studied  and  covered; 

•  The  designer— characteristics,  working  methods,  use  of  information, 
etc. 

•  The  activity— structure,  systematic  aspects  and  creativity,  factors 
affecting  design  processes,  etc. 

•  The  object  to  be  designed— nature,  properties,  processes  to  be 
performed,  necessary  parts,  abstractions  and  models,  life  stages, 
representations,  evaluations  and  decisions,  etc. 

•  The  context  in  which  engineering  design  takes  place,  and 

•  The  context  of  use  of  the  resulting  technical  system. 

Current  developments  in  attempts  to  create  design  science  are  briefly 
described.  In  particular,  reference  is  made  to  relevant  developments  in 

•  Germany  (schools  of  Prof.  Pahl  and  Beitz,  and  Prof.  Roth,  social 
context  by  Prof.  Ropoh.,  and  the  recommendations  issued  by  Verein 
Deutscher  Ingenieure), 

•  Poland  (schools  of  Dr.  Gasparski,  and  Dr.  Dietrych), 

•  Japan  (schools  of  Prof.  Yoshikawa), 

•  Switzerland  (school  of  Dr.  Hubka),  and 

•  England  (Design  Research  Society,  and  various  fragments). 

Design  science  is  brought  into  connection  with  necessary  developments  of 
some  computer  aids,  expert  systems,  and  concepts  of  CAD,  CAD/CAM, 
and  CIM." 

Finkelstein,  L.  and  A.C.W.  Finkelstein,  "Review  of  Design  Methodology,"  I  EE 
Proceedings,  Vol.  130,  Pt.  A,  No.  4,  June  1983,  pp.  213-222. 

ABSTRACT;  "This  paper  surveys  design  methodology,  the  science  of 
methods  of  design.  It  discusses  the  aims  of  design  methodology  as  well  as 
objections  to  it.  The  various  sources  of  design  methodology  are  reviewed. 

The  nature  and  structure  of  the  design  process  are  outlined.  An  organized 
presentation  is  given  of  methods  of  design  concept  generation.  Finally  the 
evaluation  and  decision  steps  in  design  are  briefly  analyzed.  In  conclusion 
the  authors  state  their  contention  that  design  methc^ology  is  a  useful 
contribution  to  design." 

Gasparski,  W.W.,  S.A.  Gregory,  R.  Foque,  and  A.  Strzalecki,  "Contemporary  History  of 
Design  Science,"  Paper  for  XVI  International  Congress  of  the  History  of  Science,  26 
August-3  September  1981,  Bucharest,  Romania. 

Right,  T.K.,  L.  Ginszauskas,  and  D.  MacLean,  "Investigation  into  the  Methods  Students 
Use  to  Solve  Mechanical  Design  Problems,"  Proceedings  of  the  International  Conference 
on  Engineering  Design,  August  1987,  Boston,  Massachusetts,  pp.  888-897. 


Prior  to  describing  research  into  the  manner  in  which  students  approach 
design  problem-solving,  the  author  summarizes  past  research  endeavors  that 
have  resulted  in  significant  contributions  to  the  study  of  engineering  design, 
regardless  of  design  discipline. 

Lera,  Sebastian.,  "Synopsis  of  Some  Recent  Published  Studies  of  the  Design  Process  and 
Designer  Behavior,"  Design  Studies,  Vol.  4,  No.  2,  April  1983,  pp.  133-140. 

This  series  of  synopses  describes  research  into  the  problem  solving 
behavior  exhibited  by  design  engineers  (architecture,  urban  planning,  civil 
engineering).  Results  from  the  following  projects  are  reviewed: 


Published 


Research 

Institution 

Techniques 

Eastman 

Carnegie  Mellon 

Lab  Observation 

Foz 

MIT 

Lab  Observation 

Cornforth 

Royal  College  of  Art 

Protocol  Analysis  and  Repertory  Grid 

Thomas  and  Carroll  * 

IBM  Research  Lab 

Lab  Experiments 

Simmonds 

Oxford  Polytechnic 

Lab  Observation 

Lera 

Royal  College  of  Art 

Scaling  Method 

Krauss  and  Myer 

MIT 

Case  Study 

Hykin 

Imperial  College 

11  Case  Studies 

Mallen  and  Goumain 

Royal  College  of  Art 

Repertory  Grid  and  Field  Studies 

Willey  and  Yeomans 

University  of  Liverpool 

Case  Studies 

Wang 

Portsmouth  Polytechnic 

Participant/Observation 

Bessant  and  McMahon 

Participant/Observation 

Darke 

University  of  Sheffield 

Interviews 

Al,  Wareh  and  Murta 

University  of  Sheffield 

Interviews 

Murtha 

University  of  Wisconsin 

Design  Exercises 

Thompson  and  Wood 

PSA,  Royal  College  of  Art 

Design  Exercise 

Libardi,  Eugene  C.,  John  R.  Dixon,  and  Melvin  K.  Simmons,  "Computer  Environments 
for  the  Design  of  Mechanical  Assemblies:  A  Rese;irch  Review,"  Received  from  the 
authors. 

The  authors  describe  significant  results  and  interesting  ideas  from  recent 
"research  in  artificial  intelligence  and  mechanical  engineering  that  is  relevant 
to  aspects  of  conceptual  design  environments  for  mechanical  systems  and 
assemblies."  This  review  is  organized  in  categories  corresponding  to  five 
basic  requirements  (defined  by  the  authors)  of  a  conceptual  design  aid  for 
mechanical  systems.  The  categories  are 

•  Methods  for  representing  top-down  design  and  multiple  viewpoints 

•  Methods  for  representing  and  using  functional  knowledge 

•  Methods  for  representing  spatial  relationships 
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Methods  for  maintaining  consistency  and  handling  inconsistency 
among  representations 

Methods  for  providing  analysis  and  other  support. 


Magee,  Kevin,  "The  Elicitation  of  Knowledge  from  Designers,"  Design  Studies,  Vol.  8, 

No.  2,  April  1987,  pp.  62-69. 

The  author  characterizes  design  activity  and  describes  techniques  that  may  # 

be  useful  to  elicit  knowledge  about  the  design  activity  from  engineers. 

Knowledge  elicitation  is  defined  as  the  process  of  documenting  expert 
knowledge  by  means  of  a  dialogue  between  a  knowledge  engineer  and 
human  expert.  The  seven  techniques  reviewed  include  interviewing, 
questionnaires,  attitude  scaling,  projective  tests,  repertory  grid,  and 

observation  of  behavior.  For  each  technique  the  author  identifies  • 

advantages  and  disadvantages.  These  are  summarized  as  follows; 


Technique 

Advantages 

Disadvantages 

Interviewing 

Flexibility 

Possibility  of  bias,  inconsistencies  between  what 
people  say  they  do  and  what  they  actually  do 

Questionnaires 

Efficiency,  convenience 

Inflexible,  inconsistencies  between  what  people 
say  they  do  and  what  they  actually  do 

Attitude  Scaling 

Provides  a  general 
measure  of  individual's 
attitudes  toward  a 
subject 

Time-consuming,  does  not  provide  detailed 
information 

Protective  Testing 

Overcomes  barriers  to 
communication 

Validity,  reliability,  and  objectivity  of  information 
disputed 

Repertory  Grid  Methods 

Provides  information 
about  the  domain,  con¬ 
tentious  points,  and  how 
the  expert  categorizes 
his  knowledge 

Does  not  handle  complexity  very  well,  limited  range 
of  applicability 

Protocol  Analysis 

Results  in  much  valuable 
and  reliable  information, 
information  about  all 
aspects  of  design 
activity  can  be  elicited. 
Useful  for  eliciting 
information  that  designers 
have  difficulty  expressing 

Analysis  of  data  time-consuming,  there  is  no 
universal  way  to  analyze  protocols 

Observation  of  Design 
Behavior 

Useful  for  eliciting  infor¬ 
mation  that  designers 
have  difficulty  expressing 

The  author  concludes  by  suggesting;  "more  effort  should  be  made  to  use  # 

and  improve  techniques  other  than  interviewing." 
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Ostrofsky,  Benjamin,  Morphology  of  Design  of  Aerospace  Systems  with  Inclusion  of 
Human  Factors.,  Final  Report  under  AFOSR  Grant  No.  77-3148,  University  of  Houston, 
August  1977,  AD  A049999. 

The  research  reported  in  this  technical  report  includes  a  "review  of  the 
literature  describing  the  design  decision  structure  as  it  relates  to  the 
development  of  aerospace  systems  as  well  as  identifying  the  human 
resource  factors  relevant  to  the  design  structure."  An  annotated 
bibliography  was  prepared  as  a  result  of  the  literature  review.  Most  of  the 
bibliographic  entries  are  from  the  journal.  Human  Factors. 

Schon,  D.A.  and  L.I.  Bucciarelli,  "Design  Theory  and  Methods— An  Interdisciplinary 
Approach,"  Preprints  of  the  Design  Theory  '88  NSF  Grantee  Workshop  on  Design  Theory 
and  Methodology,  Troy,  NY,  June  1988,  pp.  2.1. 1-2.1. 9. 

The  authors  report  on  the  status  of  current  research  underway  by  the 
members  of  the  Design  Theory  and  Methods  Group  at  MIT.  This  group  is 
concerned  with  the  nature  of  design  inquiry  and  designers'  knowledge  from 
the  perspective  of  many  disciplines:  architecture,  environmental  design, 
planning  and  engineering.  The  questions  being  addressed  by  the  research 
are 

•  What  aspects  of  designing  can  be  made  computable? 

•  What  are  the  properties  of  a  computer  assistant  useful  to  designers? 


Current  research  ventures  include 


Researchers 

Focus 

Gross.  Ervin,  Anderson, 
and  Fleisher 

Developing  "constraint  manager,"  an  automated  environment  that  models 
design  as  a  search  within  constraints 

Purcell 

Evaluating  computer  environments  to  determine  what  needs  to  be  done  to 
assist  designers 

Habraken  and  Gross 

Developing  a  conceptual  framework  (concept  design  games)  intended  to 
explore  and  illuminate  important  design  ideas 

Bucciarelli 

Studying  the  design  process  in  situ  to  explore  the  character  and  consequences 
of  the  discourse  through  which  design  participants  communicate  with  each 
other 

Porter  and  Schon 

Concerned  with  the  experienced  phenomena  of  designing 

Tempczyk,  Halina,  "A  Survey  of  Research  and  Studies  on  Design,"  Design  Studies,  Vol. 
7,  No.  4,  October  1986,  pp.  199-215. 

The  authors  draw  a  number  of  conclusions  from  191  responses  to  a 
questionnaire  distributed  in  1984-85  by  the  Design  Methodology  Unit  at  the 
Department  of  Praxeology  of  the  Institute  of  Philosophy  and  Sociology  of 
the  Polish  Academy  of  Sciences.  Information  was  obtained  about  various 
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programs  in  Poland,  the  USSR,  West  Germany,  the  UK,  USA,  West 
Berlin,  Japan,  Australia,  China,  Czechoslovakia,  Canada,  Austria, 
Bulgaria,  Holland,  Hungary,  Italy,  and  Mexico.  The  most  valuable  part  of 
this  paper  is  a  comprehensive  table  of  the  basic  information  received  from 
the  respondents.  Specific  studies  mentioned  in  the  narrative  include 


Researcher 

Nation 

Focus 

Yoshikawa 

Japan 

A  general  theory  of  design  based  on  the  thinking  process 

Gregory 

UK 

Design  projects  and  processes  assuming  a  background  of  uncer¬ 
tainty.  Collects  data  from  field  studies,  lab  observations,  protocols, 
reports,  and  public  statements  made  by  designers 

Bugs 

Canada 

Ontological  features  of  artifacts  and  features  characterizing  their 
design 

Cross 

UK 

The  epistemological  foundations  of  design  knowledge 

Jones 

UK 

A  universal  humanistic  conception  of  design 

Gasparski 

Poland 

Design  knowledge  collected  from  various  disciplines 

Volkema 

USA 

The  multiplicity  of  goals  in  the  early  phases  of  strategic  planning 

Stasiak 

Poland 

Action,  treated  as  a  syndrome  of  thought  processes 

Szucs 

Hungary 

The  general  meaning  of  technical  culture 

Nalimov 

USSR 

The  world  as  a  system  of  probabilistic  concepts 

Rubin  and 
Altshuller 

USSR 

/->  theory  of  the  development  of  technical  systems 

Samuel 

Australia 

The  formulation  of  goals  connected  with  engineering  design 

Kulakowska 

1 

Poland 

Supporting  the  design  process  in  the  informatio.-^  processing  aspect 

Kerner  and 

Miller 

Poland 

The  information  needs  of  designers 

Bonkowics- 

Sittauer 

Poland 

Formal  descriptions  of  the  design  process 

Partyka 

Poland 

Model  of  the  automatic  creation  of  functional  schemes  of  technical 
objects 

Kostov 

Jungerman 

Bulgaria 

West  Berlin 

Automating  scientific  investigations,  decisionmaking  theory 

Tondl 

Czechoslovakia 

Decisionmaking  theory 

Wise 

USA 

Cognitive  processes  in  design  decision  making 
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Some  observations  made  by  the  author  include 

•  Much  attention  is  devoted  to  problems  connected  with  teaching  design 
and  textbooks  on  design  methodology. 

•  Many  researchers  are  working  on  the  methodology  of  particular  fields 
and  an  especially  large  group  are  studying  engineering  design 
methodology. 

•  Many  researchers  opt  for  a  systems  approach. 

•  Many  projects  are  devoted  to  problems  connected  with  computer-aided 
actions,  mainly  teaching  and  design. 

•  Many  researchers  are  dealing  with  problems  of  decisionmaking  as 
related  to  design. 

Stauffer,  L  .A.  and  D.G.  Ullman,  "A  Comparison  of  the  Results  of  Empirical  Studies  into 
the  Mechanical  Design  Process,"  Design  Studies,  Vol.  9,  No.  2,  April  1988,  pp.  107-114. 


Compares  six  independent  studies  that  investigated  the  mechanical  design 
process  by  studying  human  designers.  These  studies  were  each  based  on 
empirical  data.  The  studies  are  listed  below. 


Study 

Goal 

Significance 

Marples,  1961, 
Retrospective  Analyses 

Develop  an  abstract  model  of  the 
design  process 

Describes  the  design  process  as  a  decision 
tree,  a  sequence  of  critical  decisions 
leading  from  an  abstract  problem  statement 
to  the  final  specification 

Ramstrom  and  Rhenman, 
1965,  Retrospective 
Analyses 

Suggest  a  method  of  describing 
and  analyzing  the  progress  of  an 
engineering  design  project 

R&R  are  the  first  to  label  design  as  a 
problem-solving  process.  They  also  con¬ 
sider  design  a  group  activity,  which  in  turn 
interacts  with  the  entire  organization  and 
finally  with  others  outside  the  organization 

Lewis,  1981, 
Retrospective  Analyses 

Construct  an  information  process¬ 
ing  model  of  component  design 

Waldron  and  Waldron, 
1987,  Retrospective 
Analyses 

Make  observations  about  design 
during  the  conceptual  design 
phase 

Mitroff,  1967, 

Observation 

Add  to  the  understar  ling  of 
design 

Design  is  both  a  technical  and  a  behavioral 
process.  Individual  behavior  is  as  impor¬ 
tant  as  the  technical  aspects  of  design 

Stauffer  &  Ullman, 
Protocol  Analyses 

Develop  a  detailed  model  of  the 

ME  design  process  suitable  for 
intelligent  computer  tools  and  for 
the  evaluation  of  design 
strategies 

The  work  is  based  on  the  information 
processing  model  of  Newell  and  Simon, 
and  thus  considers  problem  niving  as  a 
transformation  of  the  state  of  the  design 
through  the  application  of  operators 

Dimensions  for  comparing  these  studies: 

•  Number  of  designers  working  together  on  a  single  project 

•  Of  those  working  on  the  project,  the  number  used  in  the  study 

•  The  number  of  cases  on  which  the  study  is  based 

•  Attention  given  to  designer  interaction  with  other  individuals 

•  The  purpose  of  the  design  projects 

•  The  types  of  data  gathered. 

The  six  studies  led  to  44  conclusions  describing  design  performance 
grouped  into  4  categories: 

•  Algorithmic  vs  heuristic  nature  of  design 

•  Parallel  vs  serial  development  of  solutions 

•  Technical  vs  behavioral  nature  of  design 

•  Dependence  vs  independence  of  design  on  domain  knowledge. 

The  author  notes  "...serious  study  in  design  in  general  has  occurred  only  in 
the  past  25  years." 

B.  ENGINEERING  DESIGN  RESEARCH  PROJECTS 

1 .  Methods  Used  to  Model  the  Organizational  Design  Decision  Process 
a.  Systems  Analysis 

Fulton,  R.E.,  "A  Framework  for  Innovation,"  Computers  in  Mechanical  Engineering, 
March  1987,  pp.  26-40. 

Fulton,  R.E.  and  Salley,  G.C.,  "IPAD:  A  Unique  Approach  to  Govemment/Industry 
Cooperation  for  Technology  Development  and  Transfer,"  NASA  Technical  Memorandum 
86422,  Langley  Research  Center,  June  1985. 

ABSTRACT:  "A  key  element  to  improved  industry  productivity  is  effective 
management  of  CAD/CAM  information.  To  stimulate  advancements  in  this 
area,  a  unique  joint  govemment/industry  project  designated  Integrated 
Programs  for  Aerospace-Vehicle  Design  (IPAD)  was  carried  out  from  1971- 
1984.  The  goal  was  to  raise  aerospace  industry  productivity  through 
advancement  of  computer  based  technology  to  integrate  and  manage 
information  involved  in  the  design  and  manufacturing  process.  IPAD 
research  was  guided  by  an  Industry  Technical  Advisory  Board  (ITAB) 
composed  of  over  100  representatives  from  aerospace  and  computer 
companies.  The  project  complemented  traditional  NASA/DoD  research  to 
develop  aerospace  design  technology  and  the  Air  Force's  Integrated 
Computer-Aided  Manufacturing  (ICAM)  program  to  advance  CAM 
technology.  IPAD  had  unprecedented  industry  support  and  involvement 


and  served  as  a  unique  approach  to  govemment/industry  cooperation  in  the 
development  and  transfer  of  advanced  technology.  This  paper  summarizes 
the  IPAD  project  background,  approach,  accomplishments,  industry 
involvement,  technology  transfer  mechanisms,  and  lessons  learned  from  the 
project." 

Gartz,  P.E.,  "Avionics  Development  and  Integration  System  Methods,"  IEEE  AES 
Magazine,  June  1987,  pp.  2-8. 

ABSTRACT;  "This  paper  describes  a  set  of  life  cycle  methods  that  were 
developed  in  1980  and  1981  and  used  in  the  later  phases  of  the  757/767 
airplane  programs.  They  have  been  used  as  a  framework  to  establish  and 
guide  the  introduction  of  a  wide  use  of  similar  methods  for  the  future 
avionics  of  Boeing's  next  airplane,  the  7J7.  The  methods  were  designed  to 
improve  the  communication  of  the  system's  requirements,  architecture  and 
implementation  to  a  wide  group  of  interested  parties— not  just  the 
development  engineers  but  also  the  users,  maintainers,  customers,  and 
support  organizations.  A  major  aspect  of  these  methods  is  that  they  were 
designed  to  be  used  for  systems  in  general  not  just  software  systems. 

This  paper  describes  the  background  and  goals  and  objectives  leading  to  the 
need  for  systems  engineering  methods,  describes  the  methods,  and  gives  an 
example  of  their  use.  It  concludes  with  remarks  on  steps  taken  to  introduce 
methc^s  of  this  type  on  a  wide  scale  to  support  the  next  Boeing  airplane." 

Love,  Tom,  "A  Review  of  the  Variables  Which  Influence  the  Software  Development 
Process,"  General  Electric  Report  TIS  761SP001,  AD  #B063727,  June  28,  1976. 

ABSTRACT;  "The  software  development  process  is  divided  into  three 
stages; 

1 .  Task  demands 

2 .  Program  development  process 

3 .  Operating  program 

The  Program  Development  Process  is  further  divided  into  four  components; 
Programmer,  Computer  System,  Work  Environment,  and  Software 
Development  System. 

Within  this  framework,  recent  work  in  software  engineering  and 
psychology  are  reviewed  and  discussed.  Most  of  the  general  papers  in  the 
field  of  software  engineering  are  included  in  this  review." 

General  Electric  Space  Division. 


b.  Participant  Observation  Studies 


Bucciarelli,  L.L.,  "Reflective  Practice  in  Engineering  Design,"  Design  Studies.  Vol.  5,  No. 
3,  July  1984,  pp.  185-190. 
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The  author  describes  research  "to  develop  a  better  understanding  of  the 
design  process  and  to  determine  how  values  inform  decisions  and  thus 
affect  the  forms  of  technology  that  emerge  from  that  process."  The  method 
used  during  this  study  is  called  participant/observation.  This  method 
reveals  "the  social  as  well  as  technical  elements  of  decisions  made  in  a 
firm's  endeavor  to  develop  and  market  its  wares." 

From  this  study  the  author  has  observed: 

"Different  participants  in  the  design  process  have  different 
perceptions  of  the  design.. .The  task  of  design  is. ..as  much  a 
matter  of  getting  different  people  to  share  a  common 
perspective,  to  agree  on  the  most  significant  issues,  and  to 
shape  consensus  on  what  must  be  done  next,  as  it  is  a  matter 
of  concept  formation,  evaluation  of  alternatives,  costing  and 
sizing..." 

School  of  Engineering,  MIT. 

Bucciarelli,  L.L.  and  D.S.  Schon,  "Generic  Design  Process  in  Architecture  and 
Engineering:  A  Dialogue  concerning  at  least  Two  Design  Worlds,"  Proceedings  of  the 
NSF  Workshop  on  the  Study  of  the  Design  Process,  February  1987,  Oakland,  CA,  pp. 
295-313. 

The  authors  describe  their  individual  theories  of  how  designers  design  from 
the  points  of  view  of  architectural  design  and  engineering  design  .  As  stated 
by  the  authors: 

"Our  purpose  has  been  to  explore  a  perspective  on  design  in 
both  architecture  and  engineering  that  sees  design  as  a 
dynamic  process,  one  in  which  the  designer  shapes  and 
imagines,  calculates  and  trades  off  within  a  design  world 
which  is  personal  though  culturally  shared..." 

Wallace,  K.M.,  "Studying  the  Engineering  Design  Process  in  Practise,"  Proceedings  of  the 
1987  Congress  on  Planning  and  Design  Theory,  American  Society  of  Mechanical 
Engineers,  Boston,  MA.,  August  1987,  pp.  29-34. 

The  author  addresses  the  fundamental  question:  How  can  the  performance 
of  design  teams  be  improved?  He  describes  four  research  methods  suitable 
for  studying  the  engineering  design  process  and  presents  results  from  a 
four-year  study  of  the  design  process,  the  Cambridge  Project.  The  research 
methods  described  by  the  author  are:  case  history,  direct  observation, 
participant  observation,  and  action  research.  These  are  methods  for 
gathering  both  quantitative  and  qualitative  data.  According  to  the  author: 

"All  have  an  important  role  to  play.  If  design  researchers  are 
to  discover  links  and  relationships  between  the  different 
categories  of  knowledge,  they  must  understand,  use,  and 
adapt  the  research  techniques  from  such  diverse  fields  as 
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engineering,  sociology,  anthropology,  psychology,  manage¬ 
ment,  economics,  and  business." 


The  method  adopted  for  the  Cambridge  Project  was  participant  observation. 

The  objectives  for  this  study  included  modeling  the  design  process  observed 
in  context.  Upon  completion  of  the  project,  the  researchers  concluded  that 

"from  a  single  investigation,  or  even  a  few,  it  is  not 
reasonable  to  expect  significant  general  conclusions  or 
guidelines  to  emerge.  But.. .much  can  be  learned  about 
gathering,  verifying,  categorizing  and  analyzing  data.. ..With 
time,  as  the  number  of  investigations  builds  up,  patterns  and 
insights  will  undoubtedly  emerge.  These  will  be  valuable, 
even  if  they  cannot  ever  be  considered  as  absolute." 

University  of  Engineering  Department,  Cambridge. 

Wallace,  K.M.,  "Detailed  Analysis  of  an  Engineering  Design  Project,"  Proceedings  of  the 
International  Conference  on  Engineering  Design,  August  1987,  Boston,  MA,  pp.  94-101. 

SUMMARY:  "To  help  meet  the  calls  for  more  effective  engineering  design 
in  practice  better  understanding  of  the  engineering  design  process  in 
industry  is  needed.  Participant  observation  of  engineering  design  projects 
and  analysis  of  the  field  data  collected  is  one  possible  approach  for  this. 

The  participant  observation  of  a  two-year  engineering  design  project 
involving  37  people  and  2,368  hours  of  work  effort  is  summarized  in  this 
paper.  The  project  is  briefly  described,  and  some  of  the  quantitative  and 
qualitative  results  are  discussed  with  reference  to  a  model." 

Observations  made  by  the  author: 

•  "The  design  process  as  it  is  currently  modeled  in  theory  does  not  accurately 
match  what  happens  in  practice." 

•  "The  engineering  design  process  is  still  not  completely  understood." 
Conclusions  drawn  from  a  quantitative  analysis  of  the  data  include: 

•  "The  activity  which  accounted  for  the  highest  proportion  of  the  overall  design 
effort  was  reviewing  and  reporting  and  the  most  used  design-related  technique 
was  communicating  by  means  of  reviews  and  reports." 

•  "Over  50  percent  of  the  observed  project  effort  involved  people  working  alone 
or  in  pairs  on  specified  tasks;  30  percent  went  on  meetings  involving  two, 
three,  or  four  people;  and  9  percent  was  spent  on  letter-writing  (or  reading)  and 
telephone  calls." 

c.  Retrospewive  Analysis 

Johnson,  Edgar  M.,  "The  Role  of  Man  in  the  System  Design  Process:  The  Unresolved 
Dilemma,"  System  Design:  Behavioral  Perspectives  on  Designers,  Tools,  and 
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Organizations  (ed.  by  William  B.  Rouse  and  Kenneth  R.  Boff),  New  York:  North- 
Holland,  1987,  pp.  159-175. 

ABSTRACT:  "In  the  design  and  development  of  military  systems,  man  is 
becoming  recognized  as  a  critical  element  that  contributes  significantly  to 
system  performance.  A  number  of  trends  in  the  systems  development  and 
acquisition  process  have  contributed  to  this  view.  The  results  of  case 
studies  are  used  to  illustrate  deficiencies  in  considering  man  as  a  design 
element  and  also  to  illustrate  the  complexity  of  the  issue  of  integrating 
human  requirements  into  systems  design.  The  broad  strategy  to  consider 
man  as  a  design  element  is  simple;  the  tactics  are  complex.  Observations  are 
made  concerning  some  of  the  key  elements  influencing  the  shift  to  this 
emerging  view  of  man's  role  in  systems.  Recommendations  concerning  the 
system  development  process  and  the  field  of  human  factors  are  made  for 
translating  what  we  know  about  human  factors  into  action." 

US  Army  Research  Institute. 

Little,  S.E.,  "Incremental  and  Systemic  Innovation  Strategies:  Reflections  of  Technical 
Choice,"  Design  Studies,  Vol.  8,  No.  1,  January  1987,  pp.  41-54. 

ABSTRACT:  "This  paper  addresses  the  relationship  between  design 
strategies  and  the  range  of  alternative  technical  solutions  available  to 
designers.  The  relationship  between  systemic  and  incremental  change  is 
examined  in  the  light  of  examples  taken  from  the  aircraft  industry  and  the 
modernization  of  Britain's  railways.  The  use  of  the  term  'systemic'  rather 
than  'radical'  is  explained." 

Computing  and  Information  Studies,  Griffith  University,  Australia. 

Peterson,  James  G.,  Michael  Hagel,  Gerald  Nadler,  and  Mark  H.  Chignell,  "Aids  to  the 
Design  Process  Based  on  Techniques  used  by  Expert  Designers,"  Preprints  of  the 
Proceedings  of  the  1988  NSF  Grantee  Workshop  on  Design  Theory  and  Methodology, 
Troy,  New  York,  June  1988,  pp.  1.3.1-1.3.10. 

The  focus  of  this  project  is  the  identification  of  process  and  decision  aids  in 
design.  To  do  this,  a  structured  interview  procedure  was  developed  to 
explore  the  features  of  design  in  a  successful  real-world  design  project. 

This  technique  was  designed  to  combine  the  advantages  of  a  "think  aloud" 
method  with  a  definitive  set  of  broad-based  and  specific  questions  ranging 
from  strategic  issues  in  design  to  design  team  functions.  Tentative 
conclusions  reached  include: 

•  There  is  little  or  no  evidence  of  transportable  or  robust  process  aids  across 
engineering  design  domains. 

•  There  is  very  little  evidence  of  process  aids  in  conceptual  design  where  the 
global  mission,  purposes,  and  aims  are  transformed  into  functional 
specifications. 


•  The  assertion  that  designers  divide  and  conquer  troublesome  design  problems, 
or  decompose  goals  into  subgoals,  is  not  supported  by  the  results  of 
retrospective  case  histories. 

•  There  is  little  evidence  of  design  "reasoning"  after  the  fact. 

•  There  is  insufficient  evidence  to  reveal  and  characterize  the  "black  box"  in 
conceptual  design. 

•  Given  the  timeline  scenario  methodology,  it  was  very  difficult  to  test 
hypotheses  using  retrospective  case  studies. 

Petroski,  H.  "Design  as  Obviating  Failure,"  Proceedings  of  the  1987  International 
Congress  on  Planning  and  Design  Theory^  ed.  G.  Nadler,  Boston,  MA,  August  1987,  pp. 
49-53. 

ABSTRACT;  "The  systematic  avoidance  of  failure  is  what  characterizes 
rational  design.  Thus  historical  case  studies  of  failures  are  full  of  important 
information  for  successful  design,  and  the  designer  who  is  unfamiliar  with 
the  history  of  failures  risks  repeating  old  mistakes.  Rather  than  being 
discussed  merely  in  generalities  in  this  paper,  these  ideas  are  explained  in 
the  context  of  the  specific  structural  engineering  example  of  bridges." 

Department  of  Civil  and  Environmental  Engineering,  Duke  University. 

Promisel,  D.M.,  C.R.  Hartel,  J.D.  Kaplan,  A.  Marcus,  and  J.A.  Whittenburg,  "Reverse 
Engineering:  Human  Factors,  Manpower,  Personnel,  and  Training  in  the  Weapon  System 
Acquisition  Process,"  Technical  Report  659,  Alexandria,  VA,  US  Army  Research  Institute, 
January  1985. 

ABSTRACT:  "The  objective  of  the  Reverse  Engineering  Project  is  to 
identify  how  and  where  to  influence  the  acquisition  process  to  result  in 
effective  use  of  soldiers  in  weapons  systems.  The  project  was  initiated  by 
the  US  Army  Research  Institute  at  the  request  of  Generd  Maxwell  Thurman 
while  he  was  Deputy  Chief  of  Staff  for  Personnel.  It  was  his  position  that 
careful  examination  of  the  development  process  of  several  Army  weapon 
systems  would  provide  specific  illustrations  of  human  factors,  manpower, 
personnel,  and  training  (HMPT)  issues  and  identify  critical  events  in  the 
weapon  system  acquisition  process  (WSAP).  If  proper  attention  from  an 
HMPT  perspective  were  given  to  these  critical  events,  then  fielding  of 
operationally  useful  systems  would  be  facilitated.  Approaches  for 
accomplishing  this  would  be  developed  based  on  the  detailed  information 
acquired  from  the  study  of  the  individual  systems. 

"A  study  was  undertaken  based  on  the  "reverse  engineering"  of  four 
systems:  STINGER,  Multiple  Launch  Rucket  System  (MLRS),  BLACK 
HAWK  (UH-60A),  and  the  Fault  Detection  and  Isolation  Subsystems  of  the 
Ml  tank.  The  term  "reverse  engineering"  is  intended  to  suggest  the  process 
of  determining  how  product  of  the  WSAP  came  to  be  as  they  are.  This  is 
the  final  report  of  the  project.  It  contains  a  description  of  the  study  process, 
brief  summaries  of  the  analysis  of  the  individual  systems,  the  synthesis  of 
the  four  systems  studies,  and  the  recommendations  and  products  that  were 
developed." 
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Nutt,  P.C.,  "Sources  of  Design  Failure,"  Proceedings  of  the  1987  International  Congress 
on  Planning  and  Design  in  Management  of  Business  and  Organization,  ed.  P.C.  Nutt, 
Boston,  MA,  August  1987,  pp.  9-15. 

ABSTRACT;  "Case  studies  of  design  projects  were  analyzed  to  identify 
tactics  associated  with  failure.  These  tactics  are  applied  at  three  key  points 
in  the  design  process:  when  intentions  are  set,  ideas  are  generated,  and 
designs  are  implemented.  A  variety  of  tactics  are  used  by  designers  at  each 
of  these  points  with  some  leading  to  higher  rates  of  failure  than  others. 

Failure  was  associated  with  imposed  ideas  on  the  design  process,  adopting 
ideas  after  a  limited  search,  and  using  power  tactics  to  implement  designs. 
Suggestions  to  improve  design  practice  are  offered." 

Ohio  State  University. 

Waldron,  K.J.  and  M.B.  Waldron,  "A  Retrospective  Study  of  a  Complex  Mechanical 
System  Design,"  The  Study  of  the  Design  Process:  A  Workshop  (Coordinated  by  M.B. 
Waldron),  National  Science  Foundation  Grant  DMC  8614639,  Oakland,  CA,  February 
1987,  pp.  99-128. 

d.  Expert  System  Knowledge  Representation 

Kurumatani,  Koichi  and  Hiroyuki  Yoshikawa,  "Representation  of  Design  Knowledge 
Based  on  General  Design  Theory,"  Proceedings  of  the  International  Conference  on 
Engineering  Design,  August  1987,  Boston,  MA,  pp.  723-730. 

Describes  a  methodology  for  representing  design  knowledge  (design  objects 
and  processes)  in  a  computer  by  using  an  object-oriented  approach.  The 
authors  are  using  this  methodology  to  develop  a  "detail-design-aiding" 
system. 

The  University  of  Tokyo,  Department  of  Precision  Machinery  Engineering. 

Maher,  Mary  Lou,  "Engineering  Design  Synthesis:  A  Domain  Independent 
Representation,"  Preprints  of  the  Design  Theory  '88  NSF  Grantee  Workshop  on  Design 
Theory  and  Methodology,  Troy,  NY,  June  1988,  pp.  3.4.1  -  3.4.16. 

This  research  focuses  on  the  development  of  a  domain-independent 
knowledge-based  system  for  preliminary  design  synthesis.  The  author 
describes  design  as  a  search  process  during  which  a  design  space  is 
traversed  for  an  acceptable  solution.  To  implement  a  knowledge-based 
system  which  corresponds  to  this  view  of  design,  the  design  space  (the 
knowledge  used  during  design)  must  be  well-represented.  The  knowledge 
that  is  used  during  synthesis  is  characterized  in  terms  of  planning 
knowledge  (design  goals,  and  the  order  in  which  goals  are  pursued)  and 
design  knowledge  (alternative  solutions  for  each  goal,  constraints  on  the 
selection  of  solutions).  This  knowledge  is  represented  in  frames:  planning 
and  design  knowledge  in  hierarchically  organized  goal  frames  and  the 
design  solution  in  hierarchically  organized  component  frames. 


Gero,  J.,  M.L.  Maher,  and  F.  Zhao,  "Chunking  Structural  Design  Knowledge  as 
Prototypes,"  Preprints  of  the  Design  Theory  '88  NSF  Grantee  Workshop  on  Design 
Theory  and  Methodology,  Troy,  NY,  June  1988,  pp.  4.2.1-4.2.18. 

ABSTRACT:  "The  concept  of  a  prototype  as  a  conceptual  schema  for  the 
representation  of  generalized  design  Imowledge  is  proposed.  A  formal 
definition  of  prototypes  is  presented  independent  of  their  implementation. 

The  paper  elaborates  this  concept  and  demonstrates  its  applicability  through 
an  example  in  the  domain  of  structural  design.  The  example  of  prototype 
knowledge  is  presented  for  the  design  of  rigid  frames  and  the  process  of 
designing  using  prototypes  is  described.  The  advantages  of  this  approach 
are  presented." 

McCall,  R.  "PHIBIS;  Procedurally  Hierarchical  Issue-Based  Information  Systems." 

This  paper  describes  PHIBIS,  a  concept  which  extends  the  Issue-Based 
Information  System  (IBIS)  method.  PHIBIS  is  a  design  method  for  both 
describing  design  processes  and  attempting  to  improve  them.  The 
descriptive  aspects  of  PHlfilS  are  based  on  the  Issue-Serve  Systems 
Theory.  The  Issue-Serve  Systems  Theory  is  a  general  theory  which 
attempts  to  provide  a  theoretical  framework  for  unifying  specific  partial 
theories  such  as  theories  of  form,  decision  making,  and  participatory 
design.  The  Issue-Serve  System  Theory  is  stated  as  follows:  "a  design 
process  can  be  represented  as  a  quasi-hierarchy  of  issues  connected  by 
serve  relationships,  where  (1)  an  issue  is  any  question  arising  in  a  design 
project,  (2)  issue  A  serves  issue  B  if  answering  A  is  useful  for  answering 
B,  and  (3)  a  quasi-hierarchy  is  a  tree-like  directed  graph  structure  which 
contains  a  single  top- most  ("root")  issue  and  mostly  convergent  branches 
and  loops."  Manual  (noncomputer)  implementation  of  PHIBIS  was 
attempted  in  1979  but  proved  to  be  infeasible.  Computer  implementation 
was  begun  in  1980  in  a  program  called  MICROcomputer  Planning 
Information  System  (MICROPLIS). 

University  of  Colorado  at  Boulder,  College  of  Environmental  Design. 

Dixon,  J.R.,  and  M.K.  Simmons,  "Computers  that  Design:  Expert  Systems  for 
Mechanical  Engineers,"  Computers  in  Mechanical  Engineering,  Vol.  2,  Issue  3,  November 
1983,  pp.  10-18. 

The  authors  introduce  the  notion  of  applying  expert  system  technology  to 
the  field  of  mechanical  engineering.  An  expert  system  is  defined  as  a 
computer  program  that  has  "captured  the  experience,  knowledge,  and 
judgment  of  an  expert  practitioner  in  a  field  and  has  organized  that  expertise 
for  use  by  other  practitioners."  The  engineering  design  process  is 
characterized  as  an  ill-structured  problem  which  involves  iterative  decision¬ 
making.  Two  types  of  decisions  are  described:  technical  decisions  and 
pro^'ess  decisions. 

Dixon,  J.R.and  M.K.  Simmons,  "An  Architecture  for  Application  of  Artificial  Intelligence 
to  Design,"  in  Proceedings  of  the  21st  Design  Automation  Conference,  1984,  pp.  634-640. 
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Describes  the  decisions  designers  make  and  fundamental  characteristics  of 
the  design  process  and  relates  these  decisions  to  the  redesign  model  of  the 
design  process.  The  authors  use  the  redesign  model  as  the  basis  for  an 
architecture  for  applying  AI  to  design  and  illustrate  the  potential  use  of  the 
architecture  in  two  domains  from  mechanical  engineering. 

Dixon,  J.R.and  M.K.  Simmons,  "Reasoning  About  Quantitative  Methods  in  Engineering 
Design,"  in  Coupling  Symbolic  and  Numerical  Computing  in  Expert  Systems,  (ed.  J.S. 
Kowalik),  Elsevier  Science  Publishers,  1986,  pp.  47-56. 

Describes  research  which  "addresses  the  expert  behavior  involved  in  the 
process  of  synthesizing  a  design  for  an  engineered  object."  The  research  is 
based  on  the  redesign  model  of  the  design  process  and  focuses  on  what 
knowledge  is  used  to  refine  a  design  to  correct  its  faults  (the  redesign  step). 

This  involves  addressing  the  "kinds  of  knowledge  that  must  be  captured  in 
the  qualitative  description  of  a  quantitative  method... to  provide  an  adequate 
understanding  of  these  methods  as  regards  their  applicability  and  utility  in  a 
given  situation."  Initial  findings  suggest  that  the  "knowledge  required  for 
the  redesign  step  is  primarily  knowledge  of  the  dependencies  in  the 
problem." 

Dixon,  John  R.,  Adele  Howe,  Paul  R.  Cohen,  and  Melvin  K.  Simmons,  "Dominic  I; 
Progress  Toward  Domain  Independence  in  Design  by  Iterative  Redesign,"  Proceedings  of 
the  ASME  Computers  in  Engineering  Conference,  Chicago,  EL,  July  20-24,  1986. 

This  paper  describes  a  knowledge-based  software  implementation  of  a 
strategy  for  parametric  design  called  iterative  redesign.  Design  is  modeled 
"as  a  process  of  repeated  decomposition  into  subproblems  until  the 
subproblems  are  solvable  as  entities  without  further  decomposition." 
Iterative  redesign  "is  a  model  for  design  problem  solving  when  the  problem 
requires  no  further  decomposition  to  be  manageable  intellectually.  The 
software  implementation  of  this  strategy  is  called  Dominic.  The  knowledge 
base  consists  of  domain  specific  information  including:  problem 
specification  parameters,  design  variables,  performance  parameters, 
priorities,  satisfaction  scales,  analysis  procedures,  constraint  equations,  and 
constant  values.  Dominic  has  been  successfully  tested  in  four  different 
design  domains. 

University  of  Massachusetts,  Amherst,  Department  of  Mechanical 
Engineering,  Department  of  Computer  and  Information  Science;  General 
Electric  Corporate  Research  and  Development,  Schenectady,  NY. 

Orelup,  Mark  F.,  John  R.  Dixon,  and  Melvin  K.  Simmons,  "Dominic  II:  More  Progress 
Towards  Domain  Independent  Design  by  Iterative  Redesign,"  ASME  WAM,  December  13, 
1987,  Boston,  MA. 

Meunier,  Kenneth  L.  and  John  R.  Dixon,  "Jf-rative  Respecification:  a  Computational 
Model  for  Hierarchical  Mechanical  System  Design,"  Draft  submitted  to  the  1988  ASME 
Computers  in  Engineering  Conference. 
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The  authors  describe  a  hierarchical  computational  model  for  parametric 
mechanical  system  design  called  "iterative  respecification."  In  this  model, 
the  design  process  is  modeled  as  a  hierarchical  tree  stnicture  with  the  root 
and  interior  nodes  representing  subsystem  managers,  and  the  terminal  nodes 
representing  component  designers.  The  manager  nodes  write  problem 
specifications  for  component  designs  and  resolve  conflicts.  The  component 
designer  nodes  produce  a  component  that  meets  the  specifications. 
Computational  models  are  defined  as  models  that  can  be  used  as  the  basis 
for  domain  independent  computer  programs  that  perform  design,  but  are  not 
necessarily  an  emulation  of  how  people  do  design. 

Mitchell,  Tom  M.,  Louis  I.  Steinberg,  and  Jeffrey  S.  Shulman,  "A  Knowledge-Based 

Approach  to  Design,"  IEEE  Transactions  on  Pattern  Analysis  and  Machine  Intelligence. 

Vol.  PAMI-7,  No.  5,  September  1985,  pp.  502-510. 

This  research  focuses  on  the  developm.ent  of  an  interactive  knowledge- 
based  consultant  for  VLSI  design  (called  VEXED,  VLSI  expert  editor). 

The  VLSI  design  process  consists  of  decisions  regarding  how  to  implement 
submodules.  ^XED  is  a  prototype  of  an  implementation  of  a  model  of 
VLSI  design  which  draws  substantially  from  the  model  of  design  in  the 
MOLGEN  system.  "VEXED's  underlying  model  of  design  leaves  the  user 
with  two  types  of  decisions:  which  module  to  refine  next,  and  which 
refinement  rule  to  use."  VEXED  makes  use  of  three  categories  of 
knowledge:  knowledge  of  implementation  methods  for  generating  possible 
designs,  control  knowledge  for  guiding  the  search  for  satisfactory  designs, 
and  causal  knowledge  for  propagating  constraints  among  subparts  of  the 
designed  artifact 

Rutgers  University,  Computer-Science  Department 

Steinberg,  L.I.  and  Tom  Mitchell,  "A  Knowledge-based  Approach  to  VLSI  CAD,  The 

Redesign  System,"  Proceedings  of  the  21st  Design  Automation  Conference,  1984,  pp. 

412-418. 


This  article  describes  experiences  the  authors  had  in  developing 
RE’'  3SIGN,  "a  knowledge-based  system  T  ■  providing  interactive  aid  in 
the  tunctional  redesign  of  digital  circv'  .REDESIGN  addresses  the 
following  problem  of  functional  redesign:  Given  an  existing  circuit,  its 
functional  specifications,  and  a  desired  change  to  these  specifications,  help 
the  user  to  determine  a  change  in  the  circuit  that  will  allow  it  to  meet  the 
altered  functional  specifications  without  introducing  undesirable  side- 
effects."  REDESIGN  uses  two  separate  models  of  reasoning  to  answer 
questions  that  arise  when  redesigning  a  circuit:  causal  reasoning  and 
reasoning  about  purpose.  "In  reasoning  about  causality,  REDESIGN 
describes  both  the  behavior  and  the  specifications  for  a  data  stream... In 
reasoning  about  purposes, ...viewed  the  original  design  process  essentially 
as  a  planning  problem,  with  subgoals  derived  both  from  the  decomposition 
of  parent  goals  and  from  conflicts  between  other  subgoals."  Design 
expertise  is  represented  in  production  rules. 

Rutgers  University,  Department  of  Computer  Science. 


Steinberg,  L.I.  and  T.  Mitchell,  "The  Redesign  System:  A  Knowledge-Based  Approach  to 
VLSI  CAD,"  IEEE  Design  and  Test,  February  1985,  pp.  45-54. 

Mostow,  Jack  and  Mike  Barley,  "Automated  Reuse  of  Design  Plans,"  Proceedings  of  the 
International  Conference  on  Engineering  Design,  Boston,  MA,  August  1987,  pp.  632-647. 

The  authors  are  working  on  the  development  of  i  OGART,  a  component  of 
VEXED,  which  attempts  to  automate  the  reuse  of  design  processes  (i.e.,  the 
decisions  made  by  the  designer).  The  user  decisions  recorded  by  VEXED 
include  "which  module  to  refine  next,  and  which  refinement  rule  to  use." 

Bogart  provides  the  means  for  replaying  suitable  design  plans. 

Rutgers  University,  Computer  Science  Department. 

Brown,  David  C.  and  B.  Chandrasekaran,  "An  Approach  to  Expert  Systems  for 
Mechanical  Design,"  in  Trends  and  Applications  '83,  IEEE  Computer  Society,  NBS, 
Gaithersburg,  MD,  May  1983,  pp.  173-180. 

This  research  is  focused  on  the  design  of  mechanical  components  by 
computer.  Specifically,  the  authors  are  involved  in  a  study  of  "design 
activity  in  order  to  be  able  to  characterize  exactly  what  kind  of  design  tasks 
can  be  handled  using  AI  techniques."  The  research  has  been  limited  to  "that 
class  of  design  activity  which  is  the  most  routine,  where  a  designer  follows 
a  set  of  well-established  design  alternatives  at  each  stage  of  the  design." 

The  approach  under  development  in  this  study  involves  the  "refinement  of  a 
design  by  a  hierarchy  of  conceptual  specialists  that  solve  the  problem  in  a 
distributed  manner,  top-down,  choosing  from  sets  of  plans  and  refining  the 
design  at  each  level  of  the  hierarchy."  Each  specialist  has  different 
expertise.  Plans  consist  of  a  "collection  of  tasks  or  calls  to  specialists, 
possibly  interspersed  with  constraints.  Each  task  is  a  coherent  collection  of 
steps,  where  a  step  makes  a  single  design  decision.  This  approach  is  called 
Design  Refinement" 

In  this  research,  design  is  modeled  as  a  process  consisting  of  a  number  of 
points  at  which  alternative  possible  sequences  of  actions  may  be  pursued. 

"A  each  point,  the  choice  may  be  simple,  but  the  accumulated  effect  is  that 
of  quite  complex  problem  solving  behavior." 

An  ultimate  goal  of  this  research  is  to  understand  what  kind  of  knowledge 
structures  and  problem-solving  strategies  might  be  needed  for  more  general 
design. 

Ohio  State  University,  Department  of  Computer  and  Information  Science. 

Brown,  D.C.  and  B.  Chandrasekaran,  "An  Expert  System  for  Mechanical  Design:  A 
Progress  Report." 

This  is  an  extended  abstract  which  describes  progress  on  the  implementation 
of  a  prototype  expert  system  for  air  cylinder  design  using  the  Design 
Refinement  approach. 


Brown,  D.C.,  "Capturing  Mechanical  Design  Knowledge,"  1985,  pp.  121-129. 

This  article  describes  a  language  called  Design  Specialists  and  Plans 
Language  (DSPL),  developed  in  concurrence  with  research  on  the  design 
refinement  approach.  It  was  "developed  in  order  to  implement  the  AIR- 
CYL  system.  It  provides  a  way  of  writing  declarations  of  Specialists, 

Plans,  Tasks,  Steps,  Constraints,  Failure  Handlers,  Redesigners, 
Sponsors,  and  Selectors,  allowing  the  user  to  specify  the  knowledge 
contained  in  them." 

Worcester  Polytechnic  Institute,  Computer  Science  Department. 

Brown,  David  C.  and  B.  Chandrasekaran,  "Knowledge  and  Control  for  a  Mechanical 
Design  Expert  System,"  IEEE  Expert,  1986. 

Makes  the  assertion  "a  large  unstructured  collection  of  rules  clearly  lacks 
validity  as  a  realistic  model  of  design  because  reducing  all  knowledge  to  a 
single  form  does  not  recognize  that  there  are  many  different  types  of 
knowledge  used  in  any  design  problem-solving  activity." 

Kalay,  Y.E.,  L.M.  Swerdloff,  and  A.C.  Harfmann,  "A  Knowledge-based  Approach  to 
Dynamic  Computer-Aided  Design  Task  Allocation,"  Proceedings  of  the  Conference  on 
Planning  and  Design  in  Architecture,  Vol.  100239,  Boston,  MA,  August  1987. 

This  research  focuses  on  the  development  of  fundamental  paradigms  of 
design  as  a  computable  process.  The  basis  of  the  research  is  Simon's  state- 
transition  model  of  problem-solving  where  states  represent  solutions  and 
transitions  represent  processes  that  produce  new  candidates  for 
consideration  as  potential  solutions.  This  model  has  been  modified  slightly 
in  that  states  represent  design  goals.  Transitions  are  defined  in  terms  of 
processes  that  map  out  design  planning  strategies  among  the  alternative 
goals.  Goals  represent  design  knowledge  by  which  candidate  design 
solutions  are  evaluated.  Thus  the  design  process  is  represented  in  terms  of 
milestones.  This  paradigm  has  been  tested  through  a  PROLOG 
implementation. 

State  University  of  New  York  at  Buffalo,  School  of  Architecture  and 
Environmental  Design. 

Velinsky,  S.A.  and  M.F.  Lembeck,  "Progress  Towards  an  Expert  System  Environment  for 
Design,"  The  Study  of  the  Design  Process:  A  Workshop  (Coordinated  by  M.B.  Waldron), 
National  Science  Foundation  Grant  DMC  8614639,  O^land,  CA,  February  1987,  pp. 
533-548. 

The  focus  of  this  research  is  on  the  development  of  an  expert  system  that 
interacts  with  analysis  packages  for  design  development  purposes.  The 
system  is  being  applied  to  the  design  of  wire  rope  based  mechanical  systems 
and  is  referred  to  as  Mechanically  Intelligent  Designer  (MIND).  MIND 
embodies  a  basic  five-step  model  of  the  design  process.  Top-level  control 
of  the  design  process  is  carried  out  by  a  special  ruleset.  Each  sub-task  has  a 
corresponding  rule-set.  When  a  rule's  consequen.  requires  an  analysis 
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package,  the  inference  engine  calls  that  package.  The  authors  have  found 
that 

•  The  realm  of  design  problems  is  so  vast  and  each  problem  is  so  specific 
that  an  all  encompassing  inference  engine  is  considerably  difficult  to 
develop.  Thus,  rather  than  having  general  rules  about  design,  our 
system  uses  domain  specific  design  rules  obtained  from  an  expert. 

•  Engineering  design  is  often  highly  analysis-based  and,  thus,  an  expert 
design  system  must  be  able  to  interact  with  analysis  packages. 

•  Hardware  limitations  can  prevent  an  expert  design  system  from 
paralleling  the  best  human  designer's  approach,  i.e.,  many  computers 
have  a  single  processor,  whereas  the  human  designers  approach  can  be 
viewed  as  multi-processor  based.  However,  this  does  not  prohibit  the 
system  from  designing,  but  merely  guides  the  form  of  the  expert 
system's  design  methodology.  The  expert  system  does  not  have  to 
mimic  the  human  exactly  to  arrive  at  similar,  if  not  better,  results." 

University  of  Wisconsin-Madison,  Department  of  Mechanical  Engineering. 

Rychener,  Michael  D.,  "Expert  Systems  for  Engineering  Design,"  Expert  Systems,  Vol.  2, 
No.  1,  January  1985,  pp.  30-44. 

Provides  a  good  overview  of  expert  system  technology  in  general  and  with 
respect  to  engineering  design  problems.  Briefly  describes  a  few 
prototypical  AI  systems  that  embody  techniques  that  can  be  applied  to 
engineering  design  problems.  An  annotated  bibliography  is  included. 

Ulrich,  Karl  and  Warren  Seering,  "A  Computational  Approach  to  Conceptual  Design," 
Proceedings  of  the  International  Conference  on  Engineering  Design,  Boston,  MA,  August 
1987,  pp.  689-696. 

This  study  focuses  on  the  conceptual  phase  of  mechanical  design  to  "better 
understand  how  new  ideas  for  mechanical  devices  can  be  generated."  The 
key  hypothesis  of  this  study  is  that  "most  new  designs  come  from 
knowledge  of  old  designs."  To  test  this  hypothesis  the  authors  are 
developing  "well-defined  procedures  which  operate  on  a  well-defined 
representation  of  a  particular  domain  of  objects  and  representations."  As  a 
starting  point,  the  authors  are  studying  novel  combination,  "the  process  of 
extracting  individual  structural  attributes  from  each  of  several  known 
devices  and  combining  these  attributes  in  novel  ways  to  create  structural 
description  of  new  devices." 

Design  knowledge  used  for  novel  combination  can  be  represented 
computationally  as  a  "network  of  functional  and  structural  attributes  linked 
by  causal  relations."  With  this  knowledge  representation  scheme,  the 
authors  "have  implemented  a  program  which  does  novel  combination  in  the 
domain  of  mechanical  fasteners."  The  authors  found  that  the  successful 
application  of  the  novel  combination  procedures  depends  on  a  fixed 
framework  for  the  form  of  the  solution  (the  structural  and  functional 
attributes  of  the  design  solution  can  be  completely  identified  and  structured 
prior  to  the  realization  of  the  solution).  When  there  is  a  fixed  framework. 
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design  knowledge  and  solutions  can  be  readily  described.  The  solution  to 
most  design  problems,  however,  cannot  now  be  described  a  priori  within  a 
fixed  framework.  Thus  the  applicability  of  novel  combination  is  limited  in 
the  domain  of  engineering  design. 

MIT,  AJ  Laboratory. 

Ward,  A.  and  Waren  Seering,  "An  Approach  to  Computational  Aids  for  Mechanical 
Design,"  Proceedings  of  the  International  Conference  on  Engineering  Design,  Boston, 
MA,  August  1987,  pp.  591-597. 

The  goal  of  this  research  is  to  "develop  programming  methodologies 
adequate  for  performing  routine  design  tasks"  on  complex  mechanical 
systems.  The  approach  taken  is  to  write  object  modules,  each  representing 
a  type  of  design  component  Each  object  contains  parameters  describing  the 
component,  implements  specific  engineering  equations  and  methods  to 
calculate  further  parameter  values,  and  exchanges  information  with  other 
objects  as  appropriate.  Design  decisions  are  made  by  passing  messages 
among  the  objects.  Results  from  the  research  suggest  that  program  objects 
can  be  used  to  represent  components  of  a  design;  writing  the  objects, 
however,  is  a  difficult  task. 

Wood,  Kristin  L.  and  Erik  K.  Antonsson,  "Engineering  Design  -  Computational  Tools  in 
the  SYNTHESIS  Domain,"  The  Study  of  the  Design  Process:  A  Worhhop  (Coordinated 
by  M.B.  Waldron),  National  Science  Foundation  Grant  DMC  8614639,  Oakland,  CA, 
February  1987,  pp.  75-95. 

The  focus  of  this  research  is  on  the  development  of  computational  tools  for 
the  preliminary  phase  of  design.  The  research  emphasizes  the  synthesis  and 
decision  components  of  engineering  design.  In  this  paper,  the  authors 
present  a  model  of  the  engineering  design  process  with  respect  to  the  actions 
of  the  engineer  in  the  process  of  design  (not  necessarily  the  thoughts  he  has 
while  designing).  This  model  is  based  on  a  decision  structure  in  the  form  of 
a  hierarchical  tree.  The  root  of  the  tree  is  the  need.  The  branches  are  the 
overall  design  choices.  This  model  serves  as  a  foundation,  but  does  not 
help  the  engineer  in  everyday  practice.  To  build  upon  the  foundation,  a 
systematic  approach  to  design  based  on  the  Becker  model  (fuzzy  set  theory) 
is  constructed.  The  authors'  results  in  a  symbolic  and  graphical 
representation  of  the  design  process  following  a  data-flow  paradigm. 

California  Institute  of  Technology,  Department  of  Mechanical  Engineering. 

2 .  Methods  Used  to  Model  the  Individual  Design  Decision  Process 
a.  Protocol  Analysis 

Ballay,  Joseph  M,  "An  Experimental  View  of  the  Design  Process,"  in  System  Design, 
Behavioral  Perspective  on  Designers,  Tools,  and  Organizations,  ed.  William  B.  Rouse  and 
Kenneth  R.  Boff,  New  York:  North  Holland,  pp.  65-82. 


This  paper  describes  ''patterns  of  problem  solving  revealed  by  experiments 
with  designers  using  traditional  methods  and  computer-aided  systems  to 
solve  a  typical  industrial  design  problem."  The  goal  of  the  research  was  to 
understand  how  the  design  process  was  affected  by  the  use  of  CAD 
systems.  During  this  study,  the  authors  observed  50  designers,  design 
students,  engineers,  and  draftsmen,  completed  a  detailed  protocol  analysis 
with  18  subjects,  and  observed  one  subject  over  20  hours  of  design  activity. 

The  results  from  the  analysis  of  the  amassed  data  suggest  basic  patterns  for 
three  kinds  of  activity  in  the  overall  design  process:  planning,  routine 
primary  processes,  and  workplace  management.  A  couple  of  observations 
about  these  patterns: 

•  "The  more  kinds  of  representation  a  designer  can  use,  the  better  able  he 
is  to  work  through  complex  problems. 

•  The  role  and  function  of  the  computer  changes  as  the  design  process 
progresses.  At  the  beginning,  it  is  a  tool  for  individual  problem 
solving.. .Toward  the  end,  it  becomes  a  tool  for  group  management  or 
communication." 

Center  for  Art  and  Technology,  Camegie-Mellon  University. 

Eckersley,  Michael.  "The  Form  of  Design  Processes:  A  Protocol  Analysis  Study."  Design 
Studies,  Vol.  9,  No.  2,  April  1988,  pp.  86-94. 

The  goal  of  this  research  is  to  use  protocol  analysis  as  a  method  to  analyze 
the  verbal  behavior  of  designers  involved  in  problem-solving.  Primary 
hypothesis  tested:  "designers  vary  significantly  in  the  nature  and  amount  of 
information  processed  during  problem-solving."  Secondary,  informal 
hypothesis:  stages  in  problem  solving  become  evident  in  the  analysis  of 
protocol  data.  Problem  domain:  interior  design. 

Significance  of  the  study.  "The  first  of  its  kind  to: 

•  Chart  design  problem-solving  behavior  over  time  across  eight  variables 

•  Utilize  specially  developed  computer  programs  to  streamline  the 
encoding  and  analysis  of  verbal  protocols 

•  Use  a  sample  of  three  encoders  to  construct  a  more  veridical 
interpretation  of  protocols 

•  Allow  for  a  modest  testing  of  experimental  hypotheses." 

The  author  defines  protocol  analysis  as  "a  research  methodology  based  on 
the  psychological  theory  of  information  processing.  Protocol  analysis 
offers  the  community  of  design  scientists  a  potentially  effective  method  for 
the  controlled  observation  and  experimental  analysis  of  design  problem¬ 
solving  behavior." 

The  author  reviews  related  literature.  According  to  this  review:  Akin 
conducted  the  first  PA  study  recorded  in  the  design  literature.  (1984) 


Design  decision  process  model:  The  authors  attempted  to  model  problem¬ 
solving  behavior  as  patterns  in  the  occurrences  of  different  types  of 
verbalizations  (literal  copy  statements,  paraphrased  statements,  inferences, 
intentions/plans,  move,  search,  specific  assessment,  general  assessment, 
and  none  of  the  above)  during  the  design  process.  These  verbalization 
categories  correspond  to  the  types  of  information  processed  during 
problem-solving.  An  analysis  of  the  protocols  revealed  no  general  patterns 
across  all  subjects.  The  authors  concluded  that  the  lack  of  verbalization 
patterns  suggests  that  "designers  vary  significantly  in  the  nature  and  amount 
of  information  processed  during  problem  solving." 

Conclusion:  "Much  work  remains  to  be  done  in  order  to  establish  Protocol 
Analysis  as  a  valid  design  research  tool.  The  potential  for  experimentally 
testing  research  hypotheses  is  great,  provided  larger  subject-samples  are 
utilized..." 

Results: 

•  Study  was  successful  in  testing  the  methodology 

•  Working  hypotheses  were  confirmed. 

University  of  Maryland,  Department  of  Design. 

Malhotra,  Ashok,  John  C.  Thomas,  John  M.  Carroll,  and  Lance  A.  Miller,  Cognitive 
Processes  in  Design,  IBM  Thomas  J.  Watson  Research  Center,  Report  RC-7082,  1978. 

This  study  was  conducted  for  the  Office  of  Naval  Research  by  the 
Behavioral  Sciences  Group  at  the  IBM  T.J.  Watson  Research  Center. 

Focus  of  the  research:  identify  and  characterize  important  design  processes: 
goal  elaboration,  design  generation,  and  design  evaluation.  Goal:  Attain  an 
understanding  of  the  kinds  of  aids  that  can  assist  the  design  process  and 
make  it  more  efficient  and  more  effective.  Research  methodology:  one 
observational  study  and  two  empirical  studies  of  the  design  process. 

Model  developed: 

•  Explicitly  allows  iteration  and  recursion 

•  Provides  criteria  for  identifying  phases  of  design 

•  Provides  a  framework  within  Ahic  h  specific  hypotheses  can  be 
empirically  tested. 

Problem-solving  paradigm  underlying  research:  "A  problem  state  is  said  to 
exist  when  a  human  has  a  goal  but  no  immediate  procedure  that  will 
guarantee  attainment  of  the  goal.  The  goal  may  be  a  satisfaction  to  be 
achieved  or  a  dissatisfaction  to  be  alleviated.  Problem-solving  occurs  in 
moving  from  a  problem  state  to  a  non-problem  state.  In  problem-solving, 
then,  a  person  begins  in  an  initial  state,  uses  transformations  that  move  him 
from  one  state  to  another,  and  ends  in  a  final  state.  Any  of  these  states  and 
transformations  may  be  well-defined  or  ill-defined. 


In  the  case  of  design  problems,  the  person  is  generally  not  forced  to  start 
from  a  specific  initial  state,  and  although  there  may  be  constraints  on  what 
may  be  used,  the  transformations  are  usually  not  limited.  In  real-world 
design  situations  the  goals  are,  typically,  fuzzy  and  poorly  articulated  and 
cannot  be  mapped  directly  into  properties  of  the  design.  Thus,  the  exact 
configuration  of  the  final  state  is  not  prescribed.  A  part  of  the  design 
process  consists  of  formalizing  and  refining  the  design  goals  into  functional 
requirements  that  can  be  matched  by  properties  of  the  design." 

Observational  study  method:  Study  "consisted  of  video-taping  actual 
problem-oriented  dialogs  between  real  people,  playing  real  roles,  and 
attempting  to  solve  real-world  problems.  The  analyses  are  based  on 
transcriptions  of  the  videotapes. ...Scenario:  a  client  (C)  confronts  a 
consultant,  expert,  or  designer  (D)  with  a  problem  and  the  expert  is 
expected  to  provide  a  solution  to  this  problem.. ..Actual  C-D  dialogs  include 
the  examination  of  (partial)  proposed  designs  to  test  if  they  violate  some 
unstated  goal,  the  domination  of  a  design  solution  by  a  better  one,  and  the 
consolidation  of  design  components  into  a  final  design." 

Observed  behavior:  "C-D  dialogs  consisted  of  a  series  of  'cycles';  each 
cycle  consisting  of  a  regular  succession  of  'states.'  A  state  is  defined  as  a 
portion  of  the  dialog  that  is  oriented  towards  a  single  purpose. ...C-D 
dialogs  are  structured  into  cycles  each  of  which  consists  of  introducing 
some  requirements,  discussing  them,  outlining  a  solution,  and  elaborating 
and  exploring  it.  The  solutions  proposed  in  a  cycle  do  not,  however, 
always  match  the  requirements  discussed  in  it  Unsatisfied  requirements  are 
carried  implicitly  into  subsequent  cycles." 

Model  of  the  design  process:  The  design  process  is  decomposed  into  three 
fundamental  underlying  processes:  go^  elaboration,  design  generation,  and 
design  evaluation. 

Implications  and  conclusions:  The  proposed  model  suggests  the 
characteristics  of  aids  to  the  various  sub-processes  within  the  design 
process: 

•  Aids  to  Goal  Elaboration:  "One  strategy  for  assisting  the  goal 
elaboration  process  is  suggested  by  the  C-D  dialogs:  Provide  a 
mechanism  for  describing  a  number  of  designs  that  may  or  may  not  be 
appropriate  to  the  client. 

•  Aids  to  design  generation:  "Should  assist  the  designer  in  generating 
design  elements  and  design  organizations  to  meet  specified  functional 
requirements. ..in  many  cases. ..the  purpose  of  the  aid  should  be  to  help 
him  retrieve  the  information...in  other  cases,  the  designer  may  have  to 
search  for  design  elements  and  organizations  in  the  outside  world." 

•  Aids  to  design  evaluation:  "The  designs  must  be  represented  in  a 
formal  manner  and  the  properties  of  the  design  derived  mathematically 
from  the  design  specifications." 


Stauffer,  L.A.,  D.G.  Ullman,  and  T.G.  Dietterich,  "Protocol  Analysis  of  Mechanical 
Engineerine  Design,"  Proceedings  of  the  International  Conference  on  Engineering  Design, 
August  1987,  Boston,  MA,  pp.  123-137. 

Stauffer,  L.A.,  David  G.  Ullman,  and  Thomas  G.  Dietterich,  "Preliminary  Results  of  an 
Experimental  Study  of  the  Mechanical  Design  Process,"  in  The  Study  of  the  Design 
Process:  A  Workshop  (Coordinated  by  M.B.  Waldron),  Oakland,  CA,  February  1987,  pp. 
157-199. 

Ullman,  David  G.,  Stephen  Wood,  and  David  Craig,  "The  Importance  of  Drawing  in  the 
Mechanical  Design  Process,"  Design  Process  Research  Group,  DPRG-89-1,  Department 
of  Mechanical  Engineering,  Oregon  State  University,  Corvallis,  OR,  January  1989. 

This  paper  is  a  study  on  the  importance  of  drawing  (both  formal  drafting 
and  informal  sketching)  in  the  process  of  mechanical  design.  Five 
hypotheses,  focused  on  the  types  of  drawings,  their  necessity  in  mechanical 
problem  solving,  and  their  relation  to  the  external  representation  medium  are 
presented  and  supported.  Support  is  through  referenced  studies  in  other 
domains  and  the  results  of  protocol  studies  performed  on  four  mechanical 
designers.  Video  tapes  of  all  the  marks-on-paper  made  by  designers  in 
representative  sections  of  the  design  process  were  studied  in  detail  for  their 
type  and  purpose.  The  resulting  data  is  supportive  of  the  hypothesis. 

These  results  also  give  requirements  for  future  computer-aided  design  tools 
and  graphics  education  and  goals  for  further  studies. 

Ullman,  D.G.,  T.G.  Diettench,  and  L.A.  Stauffer,  "A  Model  of  the  Mechanical  Design 
Process  Based  on  Empirical  Data,"  Al-EDAM,  Vol.  2.  No.  1,  1988,  pp.  33-52.. 

Describes  the  results  from  a  two-year  study  of  the  mechanical  design 
process  based  on  the  evaluation  of  experienced  mechanical  designers 
solving  real  design  problems. 

Research  Method:  Five  mechanical  design  engineers  were  given  initial 
specifications  for  one  of  two  simple  industrial  design  problems.  "Their 
verbal  reports,  sketches,  and  gestures  were  video-taped  and  audio-taped  for 
a  period  of  6-10  hours..." 

Results:  An  ongoing  analysis  of  the  data  has  lead  to  the  development  of  a 
model  of  the  design  process  called  TEAM  (task-episode  accumulation 
model).  TEAM  "explains  many  aspects  of  the  design  process  and  it 
provides  significant  insight  into  the  way  designs  are  developed.” 

"TEAM  is  a  problem-space  model  in  which  the  fundamental  components  are 
the  design  state  and  the  design  operators.  The  design  state  contains  all 
information  about  the  evolving  design  including  problem  specifications, 
additional  constraints  introduced  by  the  designer,  proposed  designs, 
drawings,  calculations,  assembly  plans,  and  so  on.  Design  operators  are 
primitive  information  processes  that  modify  the  design  state  by  performing 
calculations,  creating  new  proposed  designs,  evaluating  proposed  designs, 
and  making  decisions  to  accept  or  reject  proposed  designs.  TEAM  contains 
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10  operators:  select,  create,  simulate,  calculate,  compare,  accept,  reject, 
suspend,  patch,  and  refine." 

"To  accomplish  a  design,  the  design  engineer  applies  the  primitive  operators 
in  meaningful  sequences  called  episodes.  An  episode  is  a  sequence  of 
operator  applications  that  address  some  primitive  goal.. .Within  an  episode, 
the  decisions  about  what  operator  to  apply  next  is  guided  by  a  set  of 
heuristic  rules." 

"According  to  TEAM,  the  goal  structure  of  the  design  process  has  three 
main  levels.  At  the  highest  level  is  the  main  goal  comprising  the  satisfaction 
of  the  given  constraints.  This  top-level  goal  is  decomposed  into  finer  goals, 
each  of  which  is  solved  by  a  single  task.  These  task-level  goals  are  further 
decomposed  into  the  primitive  goals  that  are  each  addressed  by  an  episode." 

Significance  of  model:  This  overall  problem-solving  strategy  selects,  at 
each  episode,  the  alternative  that  seems  best.  "There  is  no  global  search  that 
constructs  whole  alternative  proposals  in  detail  and  evaluates  them  to  select 
the  best.  Our  designer  subjects  are  satisficers,  not  optimizers." 

For  this  strategy  to  succeed: 

(1)  Designers  must  choose  a  good  conceptual  design  during  the  early 
episodes, 

(2)  Designers  must  be  able  to  generate  and  select  good  refinements 
throughout  the  design,  and 

(3)  Designers  must  be  able  to  identify  constraint  violations  and  apply 
good  patches  to  the  design. 

This  work  is  based  on  models  of  information  processing  psychology 
developed  by  Newell  and  Simon. 

Conclusions:  "The  development.. .of  TEAM.. .has  shown  the  usefulness  of 
protocol  analysis  as  an  exploratory  tool  for  studying  the  design  process. 

There  is  still  more  protocol  analysis  can  tell  us  about  the  design  process, 
especially  in  the  area  of  identifying  the  ways  designers  represent  and  reason 
about  proposals  and  constraints." 

Ullman,  D.G.and  T.G.  Dietterich,  "Progress  in  Understanding  the  Process  of  Mechanical 
Design,"  Design  Theory  1988  NSF  Grantee  Workshop  on  Design  Theory  and 
Methodology,  Rensselaer  Polytechnic  Institute,  June  3-5,  1988,  pp.  1.1.1-  1.1.11. 

Discusses  results  of  the  NSF-sponsored  study  described  above.  The 
objectives  of  this  study  were  to 

•  Develop  a  model  of  the  design  process  from  empirical  data, 

•  Develop  representations  for  mechanical  design  data,  and 

•  Design  software  to  test  the  findings. 


The  mechanical  design  process  and  the  results  from  this  study  will  be 
examined  in  greater  detail  during  a  new  three-year  study.  One  specific  goal 
of  the  new  study  is  "to  design  and  implement  a  design  history  tool.” 


Ullm.an,  David  G.  (Department  of  Mechanical  Engineering),  Thomas  G.  Dietterich 

(Department  of  Computer  Science),  "Research  Overview,"  Mechanical  Design  Process 

Research  Group,  Oregon  State  University,  Corvallis  OR,  November  1988. 

The  Mechanical  Design  Process  Research  Group  is  a  four-year  old 
association  of  faculty  and  students  at  Oregon  State  University.  Members  of 
this  group  have  a  common  interest  in  the  study  of  the  mechanical  design 
process  and  the  potential  for  applying  computer  techniques  to  improve 
design  performance.  The  group,  led  by  Associate  Professors  David  Ullman 
and  Thomas  Dietterich,  has  had  the  involvement  of  other  faculty  in 
Mechanical  Engineering  and  Computer  Science,  along  with  faculty  in 
Industrial  Engineering  and  Psychology.  Most  notable  of  these  has  been  the 
contribution  of  Professor  Donna  Cruse  of  the  Psychology  Department. 

To  date,  15  graduate  and  5  undergraduate  students  have  taken  part  in  the 
research  effort,  with  one  granted  PhD  (now  an  Assistant  Professor  at  the 
University  of  Idaho)  and  six  completed  masters  degrees.  The  status  of 
many  current  students  in  the  group  are  presented  in  the  discussion  of  the 
current  research  efforts.  This  is  preceded  by  the  Group's  goals  and 
a£<;complishments. 

Ullman,  David  G.,  "A  Taxonomy  of  Mechanical  Design,”  Department  of  Mechanical 

Engineering,  Oregon  State  University,  Corvallis,  OR,  23  December  1988. 

In  this  paper,  a  taxonomy  for  mechanical  design  has  been  developed  and 
used  to  compare  and  contrast  differing  research  and  mechanical  design  tool 
developments.  It  has  the  potential  of  forming  a  basis  for  classification  for 
all  mechanical  design  research.  Specifically,  if  all  proposals  for  research  on 
the  mechanical  design  process  had  to  have  a  section  where  the  effort  was 
classified  according  to  a  taxonomy  such  as  that  proposed  here,  then 
evaluating  the  proposals  may  be  made  much  easier.  At  least  reviewers 
would  have  an  initial  guide  on  how  to  view  the  research.  If  the  proposers 
found  that  the  taxonomy  was  too  constrictive  to  allow  for  adequate 
classification,  then  they  could  state  why  and  propose  a  change  to  the 
taxonomy.  Research  reports  on  mechanical  design  process  could  also 
classify  themselves  accordingly. 

Additionally,  the  taxonomy  may  be  useful  in  accrediting  academic 
programs.  Within  academia,  the  term  "design"  has  always  been 
ambiguous.  As  part  of  an  accredited  undergraduate  curriculum,  there  must 
be  a  significant  portion  of  design  in  the  curriculum.  However,  this  has  been 
hard  to  measure,  much  less  enforce.  It  is,  however,  possible  to  define  a 
clearer  and  more  measurable  definition  of  design  using  the  taxonomy.  For 
example:  Courses  that  rely  solely  on  parametric  techniques  (in  terms  of 
transforming  from  artifact  types  to  specific  instances),  such  as  mechanical 
component  design  and  most  fluid-thermal  design  courses,  could  be 
considered  Category  1  courses.  Courses  that  develop  perceived  need  to 
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concepts  could  be  Category  2  courses.  The  accreditation  requirement  could 
be  for  credit  in  both  Category  1  and  Category  2  courses.  The  gradation 
could  be  made  even  finer  if  desired. 

It  must  be  reiterated  that  this  taxonomy  is  seen  as  a  first  step.  It  is  hoped 
that  the  community  adopts  it  and  refines  it  as  the  discipline  of  mechanical 
design  becomes  better  defined  and  more  of  a  formal  science. 

b.  Laboratory  and  Computational  Experiments  of  Problem  Solving 
Behavior 

Dyer,  Michael,  G.  and  Margot  Flowers,  "Automating  Design  Invention,"  Autofact  6 
Conference  Proceedings,  October  1-4,  1984,  Anaheim,  CA,  pp.  25.1-25.21. 

The  authors  are  "interested  in  understanding  all  of  those  cognitive  skills 
which  engineers  possess,"  and  are  addressing  the  application  of  AI  research 
to  design.  The  specific  focus  of  the  research  is  formalizing  rules  of 
creativity  in  the  area  of  mechanical  engineering  design.  In  this  paper,  the 
authors  show  how  rules  of  invention  may  be  used  in  mechanical  design. 

This  research  is  implemented  in  an  "intelligent  design  invention  system 
(IDIS). 

University  of  California  at  Los  Angeles. 

Design  Process  Model:  "The  process  of  invention... is  viewed  as  a  search 
through  a  large  space  of  states  representing  objects  or  ideas.  Movement 
from  one  state  to  another  is  accomplished  by  the  application  of  an  operator. 

In  our  case,  the  state  space  consists  of  object  designs  and  the  operators  are 
"creativity  rules"  or  rule  schemas  which  modify  or  combine  existing  objects 
to  yield  new  designs." 

Findler,  Nicholas,  V.,  "Analogical  Reasoning  in  Design  Processes,"  Design  Studies,  Vol. 
2,  No.  1,  January  1981,  pp.  45-51. 

This  research  is  based  on  the  contention  that  design  is  an  "ill-structured" 
problem  and  thus  is  difficult  to  solve  by  any  set  algorithm.  Preliminary 
studies  focus  on  the  role  of  analogical  reasoning  in  design  problem  solving 
by  computers.  Specifically  addresses  the  problem  of  what  information 
should  be  extracted  from  raw  experience  consisting  of  descriptions  of 
problems  and  solutions,  and  how  this  information  can  then  be  used  in 
determining  solutions  to  new  problems.  This  paper  describes  how  certain 
programmable  procedures  (whose  cognitive  strategies  are  based  on  two 
different  models  of  analogical  reasoning)  can  transform  raw  experience  in 
design  problem  solving  into  the  knowledge  base  of  an  intelligent  system. 

The  State  University  of  New  York  at  Buffalo,  Department  of  Computer 
Science. 
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Fischer,  Gerhard  and  Boecker  Heinz-Dieter,  "The  Nature  of  Design  Processes  and  How 
Computers  Can  Support  i  nem,"  European  Conference  on  Integrated  Interactive  Computing 
Systems,  Stresa,  Italy,  September  1982. 

The  authors  are  conducting  research  to  improve  man-machine 
communication  through  the  creation  of  computer  systems  for  cooperative 
problem-solving  between  man  and  machine.  The  particular  focus  is  on 
systems  which  support  design  processes  in  different  domains.  Necessary 
preconditions  for  the  design  of  these  systems  requires  an  understanding  of: 

( 1 )  the  boundaries  between  domain-specific  and  domain  independent  design 
knowledge;  (2)  what  a  designer  does;  and  (3)  what  type  of  difficulties  a 
designer  encounters.  The  research  concepts  are  being  implemented  in  a 
knowledge-based  system  called  Information  Manipulation  System 
(INFORM).  INFORM  "incorporates  the  idea  of  a  symbiotic  environment  of 
man  and  machine  which  combines  the  advantages  of  both  to  cooperatively 
solve  design  tasks  which  neither  of  them  could  solve  as  quickly  and  as 
easily  alone."  The  knowledge  base  uses  an  object-oriented  representation 
language. 

University  of  Stuttgart,  Computer  Science  Department. 

Frey,  Claire  E.,  Gerald  Nadler,  and  Joseph  M.  Smith,  "Problem  Formulation  Methods  in 
Engineering  Design,"  The  Study  of  the  Design  Process:  A  Workshop  (Coordinated  by 
M.B.  Waldron),  Oakland,  CA,  February  1987,  pp.  203-232. 

ABSTRACT;  "This  proposal  hypothesizes  that  engineers  can  significantly 
improve  their  individual  performances  by  adopting  explicit  methods  found 
to  be  effective  in  taking  a  problem-as-stated  and  converting  it  into  the  'right' 
problem  on  which  to  work.  The  specific  objectives  of  this  reseaich  are  to: 

(1)  Identify  the  techniques  and  tools  used  by  successful  engineers  in 
developing  a  statement  of  the  problem  on  which  to  work, 

(2)  Experimentally  evaluate  various  techniques  for  arriving  at  an  effective 
problem  statement, 

(3)  Provide  prescriptive  guides  (and  computer  aids)  for  the  problem 
formulation  phase  of  design  which  can  be  adopted  by  practicing 
engineers,  and 

(4)  Make  available  computer-recorded  real-world  problems  and 
information  which  may  be  useful  for  student  engineering  education 
and  perhaps  eventually  for  professional  engineering  examinations." 

Green,  Douglas  S.and  David  C.  Brown,  "Spatial  Reasoning  During  Design:  Fit  and 
Assembly,"  Proceedings  of  the  International  Conference  on  Engineering  Design,  August 
1987,  Boston,  MA,  pp.  723-730. 

Research  is  directed  towards  understanding  the  knowledge  used  and 
problem-solving  behavior  exhibited  during  routine  design  tasks. 
Specifically  addresses  the  question  of  how  a  designer  determines  the  content 
of  a  constraint  with  respect  to  how  well  previously  designed  sub¬ 
components  fit  together.  This  spatial  reasoning  process  is  characterized  as  a 
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qualitative,  sequential,  four-stage  process.  Each  stage  represents  a  different 
set  of  decisions  that  must  be  made  about  the  fit.  The  authors  have 
implen’cnted  two  stages  of  this  model  of  spatial  reasoning  and  have  applied 
i^  to  a  simple  example.  Extensions  are  required  to  apply  the  model  to  more 
complex  and  realistic  objects.  Ultimate  goal:  understanding  how  design 
knowledge  is  compiled.  This  work  is  an  outgrowth  of  the  Brown  and 
Chandrasekaran  research  on  routine  design. 

Artificial  Intelligence  Research  Group,  Department  of  Computer  Science, 
Worcester  Polytechnic  Institute. 

Gross,  Mark  and  Aaron  Fleisher,  "Design  as  the  Exploration  of  Constraints,"  Design 
Studies,  Vol  5,  No.  3,  July  1984,  pp.  137-138. 

The  authors  provide  philosophical  insights  into  knowledge-based  systems 
for  design  and  briefly  describe  work  on  a  computer  program  which 
embodies  the  notion  of  "design  as  a  society  of  experts."  The  motivation  for 
the  research  is  described  by  the  authors: 

"We  want  to  learn  how  to  think  about  design— as  architects 
do  it.  We  have  an  idea  or  two.  We  propose  to  make  of  them 
a  computer  program.  If  the  computer  then  acts  a  bit  like  an 
architect,  then  the  experiment  will  have  succeeded.  We  shall 
be  pleased.  Our  theory  survived;  the  program  is  worth  a 
round  of  elaboration. 

The  program  is  at  once  a  theory  and  an  experiment.  As  an 
experiment,  it  has  the  advantages  of  clarity  and  control.  As  a 
theory,  it  is  disadvantaged  by  distance.  All  blessings  are 
mixed." 

The  authors  describe  the  difference  between  the  early  diagnostics-oriented 
expert  systems  which  reason  with  data  and  facts  and  design  programs: 

"The  design  program  proceeds  from  preference,  modulated  by  context  and 
circumstance.  It  is  headed  by  constraints." 

The  process  of  design  is  described  as  "an  exploration  of  fixes  that  meet  the 
constraints." 

Massachusetts  Institute  of  Technology. 

Habraken,  John  N.,  "Concept  Design  Games,"  The  Study  of  the  Design  Process:  A 
Workshop  (coordinated  by  M.B.  Waldron),  Oakland,  C  A,  February  1987,  pp.  133-154. 

Hight,  T.K.,  L.  Ginszauskas,  and  D.  MacLean,  "Investigation  into  the  Methods  Students 
Use  to  Solve  Mechanical  Design  Problems,"  Proceedings  of  the  International  Conference 
on  Engineering  Design,  August  1987,  Boston,  MA,  pp.  888-897. 

ABSTRACT:  "Two  design  exeicises  have  been  given  to  three 
undergraduate  engineering  classes  to  investigate  students'  approach  to 
solving  design  problems.  Upperclassmen  were  more  organized  and 


thorough  in  response  to  an  unstructured  problem,  and  displayed  a  better 
understanding  of  the  problem  and  a  more  complete  view  of  the  design 
process  during  a  structured  exercise." 

Department  of  Mechanical  Engineering,  Santa  Clara  University. 

Kerley,  James,  J.,  "Retroduction;  A  New  Structured  Approach  to  Mechanical  Design," 
Proceedings  of  the  International  Conference  on  Engineering  Design,  August  1987,  Boston, 
MA,  pp.  123-137. 

Pask,  Gordon,  The  Influence  of  Learning  Strategy  and  Performance  Strategy  upon 
Engineering  Design,  System  Research  LTD,  Surrey,  England,  Progress  Reports  3-7, 
1976-1977. 


These  progress  reports  describe  a  study  performed  for  AFOSR  by  System 
Research,  Ltd.,  to  evaluate  the  influence  of  cognitive  style  upon  design 
ability  in  the  domain  of  electronic  engineering  design.  The  main  hypotheses 
under  test  in  this  study  are: 

Designers  who  exhibit  high  versatility  scores  on  the  stylistic 
tests  will  produce  better  rated  designs.  Versatility  is  a 
reliable  index  of  potential  design  creativity. 

The  process  of  design  will  reveal  a  greater  use  of  valid 
analogical  reasoning  on  the  part  of  designers  deemed  "good" 
on  any  of  these  criteria. 

The  research  methodology  is  described  as  follows: 

Designers  from  two  groups  (experienced  designers  and  students)  are  pre¬ 
tested  for  learning  style  and  scored  in  terms  of  operation  learning, 
comprehension  learning,  and  versatility.  All  subjects  engage  in  a  many 
session  design  task.  The  resulting  designs  are  evalu  ated.  Behavior  during 
the  design  task  is  recorded  conventionally  (sketches,  comments,  etc.)  and 
through  a  computerized  interrogation  program  with  which  the  designer 
records  justifications  and  quantifiable  explanations  for  his  design.  After 
completing  individual  assignments,  the  subjects  are  assembled  into  groups 
for  a  group  design  task.  The  object  of  the  group  sessions  is  to  sample 
communication  between  designer,  their  activity  as  a  team,  and  the  influence 
of  personal  style  and  design  performance  upon  their  interaction.  Dialog 
during  group  design  is  tape-recorded  and  content  analyzed. 

Preliminary  results  from  the  individual  design  sessions  suggest  that 
analogical  reasoning  is  ubiquitous.  The  use. ..of  analogy  is  frequent  and,  in 
contrast  to  exponents  of  an  ^gorithmic  model  of  design,  a  prominent  part  of 
both  student  and  expert  behavior. 

Waldron,  M.B.,  W.  Jelinek,  D.  Owen,  and  K.J.  Waldron,  "A  Study  of  Visual  Recall 
Differences  Between  Expert  and  Naive  Mechanical  Designers,"  Proceedings  of  the 
International  Conference  on  Engineering  Design,  August  1987,  Boston,  MA,  pp.  86-93. 
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The  purpose  of  this  research  is  to  develop  a  deeper  understanding  of  the 
conceptual  design  process  to  design  computer  aids  for  conceptual  design. 
This  study,  in  particular,  focuses  on  visual  and  spatial  reasoning  skills  used 
by  mechanical  engineers  during  design  conceptualization.  The  hypothesis 
under  test  was  that  as  experience  in  design  increases,  the  efficiency  with 
which  the  designer  copes  with  information  should  also  increase.  The 
experiment  consisted  of  presenting  ME  drawings  to  three  groups  of 
subjects.  Each  subject  was  asked  to  look  at  each  drawing  and  to  reprc^uce 
it  from  memory  on  a  sheet  of  paper.  The  total  number  of  references  to  the 
drawing  and  the  average  time  between  references  were  recorded.  Each 
subject  was  asked  to  think  aloud  while  completing  the  task.  Task 
performance  protocols  were  also  recorded  for  each  use. 

The  authors  conclude  that  the  results  of  the  study  "support  the  original 
hypothesis  that  the  efficiency  of  information  handling  by  the  mechanical 
designers  is  dependent  on  their  level  of  expertise.. ..Further,  higher  order 
information  is  dealt  with  at  a  higher  symbolic  level  by  the  experts  than  by 
the  naive  designers.  The  naive  seem  to  focus  more  on  lines  and  sizes 
whereas  experts  focus  on  the  features  resulting  in  fewer  errors,  references, 
and  longer  drawing  time  between  references." 

C .  OTHER  SOURCES  OF  INTEREST 


1 .  Design  Theory 

Alexander,  E.R.,  "Design  in  Planning  and  Decision  Making:  Theory  and  Its  Implications 
for  Practice." 

ABSTRACT:  "Definitions  of  design  are  reviewed,  and  three  models  of  the 
design  process  are  presented.  One  definition  sees  the  design  as  roughly 
equivalent  to  problem-solving  or  decision  making,  but  in  the  context  of  the 
design  professions.  The  alternative,  "generic"  definition  sees  design  as  the 
development  of  alternative  courses  of  action.  Three  models  of  the  design 
process  are:  I.  Design  as  systematic  search  and  information  processing;  II. 

Design  as  intuitive  creativity;  HI.  An  integrative  process  combining  Models 
!  and  II.  The  intrinsic  contradictions  between  Models  I  and  II  are 
discussed.  These  imply  the  inadequacy  of  Model  III  and  demand  a  more 
genuine  synthesis.  Directions  for  developing  such  a  synthesis  are 
suggested,  and  the  adoption  of  the  generic  definition  of  design  is  advocated. 
Implications  for  design  education  are  a  mixed  curriculum  including  both 
systematic  and  experiential  teaching  formats.  Implications  for  practice  are 
the  lack  of  short-term  answers,  but  the  promise  of  an  ultimately  useful 
model  of  the  design  process." 

Cha-  drasekaran,  B.,  "Design:  An  Information  Processing-Level  Analysis,"  Draft  Version 
o^"  1  ‘':ipter  2  of  Design  Problem  Solving:  Knowledge  Structures  and  Control  Strategies, 
t.  itate  University  Technical  Report,  January  1988. 

^  BSTRACT:  "Design  problem  solving  is  analyzed  as  an  information¬ 
processing  task:  the  task  and  its  information  requirements  are  analyzed. 


This  analysis  suggests  possible  decompositions  of  the  task  into  a  number  of 
subtasks,  depending  upon  what  kinds  of  knowledge  are  in  fact  available  in  a 
domain.  This  decomposition  can  be  carried  on  several  levels  until  we  reach 
an  understanding  of  how  various  generic  problem-solving  capabilities  can 
come  together  to  help  solve  the  design  problem.  This  analysis  suggests 
possible  problem  solving  architectures  for  design.  A  number  of  AI 
approaches  to  design  are  discussed  in  this  perspective  and  it  is  shown  how 
each  of  them  can  be  understood  as  solving  a  particular  version  of  the  design 
problem,  using  one  of  the  architectures  that  arises  from  the  analysis  in  such 
a  way  that  the  architecture  matches  the  knowledge  available  in  the  domain." 

Christakis,  Alexander  N.,  David  B.  Keever,  and  John  N.  Warfield,  "Development  of 
Generalized  Design  Theory  and  Methodology,"  in  The  Study  of  the  Design  Process:  A 
Workshop  (Coordinated  by  M.B.  Waldron),  Oakland,  CA,  February  1987,  pp.  3-72. 

Gasparski,  W.W.,  "Design  Skills:  How  to  Find  Them?,"  May  1980. 

ABSTRACT:  "After  considering,  from  a  methodological  point  of  view,  an 
'ideal'  procedure  of  design,  a  concrete  application  is  proposed  in  the 
identification  of  design  skills.  Three  types  of  design  skill,  associated  with 
reception,  transformation,  and  communication  of  design  information,  are 
analyzed  in  the  form  of  a  'skills  tree.'  The  paper  ends  with  remarks  on  the 
importance  of  design  education  and  on  the  guidelines  for  designing  a  design 
course." 

Department  of  Praxiology,  Polish  Academy  of  Sciences,  Warsaw. 

Gregory,  S.A.,  "Expert  Systems  Versus  Creativity  in  Design,"  Proceedings  of  the 
International  Conference  on  Engineering  Design,  August  1987,  Boston,  MA,  pp.  615-621. 

Provides  a  historical  perspective  of  the  study  of  the  role  of  creativity  in 
design  with  a  particular  emphasis  on  the  contributions  of  expert  system 
technology.  The  author  raises  some  important  questions  regarding  the 
development  of  suitable  models  of  design  that  can  be  incorporated  into 
expert  systems. 

Lindhult,  M.S.,  "Towards  an  Intuitive  Computer-Aided  Design  Process,"  Proceedings  of 
the  Conference  on  Planning  and  Design  in  Architecture,  Vol.  100239,  Boston,  MA, 
August  1987. 

Describes  improvements  needed  to  make  CAD  systems  "more  responsive  to 
designers  so  that  the  computer  will  fit  intuitively  into  the  design  process." 

This  includes  a  summary  of  the  major  issues  concerning  the  integration  of 
expert  systems  with  CAD. 

University  of  Massachusetts,  Amherst,  Department  of  Landscape 
Architecture  and  Regional  Planning. 

Mostow,  J.,  "Toward  Better  Models  of  the  Design  Process,"  The  AI  Magazine,  Spring 
1985,  pp.  44-57. 
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ABSTRACT:  "What  are  the  powerful  new  ideas  in  knowledge  based 
design?  What  important  research  issues  require  further  investigation? 

Perhaps  the  key  research  problem  in  Al-based  design  for  the  1980s  is  to 
develop  better  models  of  the  design  process.  A  comprehensive  model  of 
design  should  address  the  following  aspects  of  the  design  process: 

•  The  state  of  the  design 

•  The  goal  structure  of  the  design  process 

•  Design  decisions 

•  Rationales  for  design  decisions 

•  Control  of  the  design  process 

•  The  role  of  learning  in  design. 

This  article  presents  some  of  the  most  important  ideas  emerging  from 
current  AI  research  on  design,  especially  ideas  for  better  models  of  design. 

It  is  organized  into  sections  dealing  with  each  of  the  aspects  of  design  listed 
above." 

Nadler,  Gerald,  "Systems  Methodology  and  Design,"  Transactions  on  Systems,  Man,  and 
Cybernetics,  Vol.  SMC- 15,  No.  6,  December  1985,  pp.  685-697. 

ABSTRACT;  The  problems  facing  the  economy  and  its  productivity  level 
and  rate  of  change  have  many  sources.  One  is  related  to  engineering 
problem  solving  and  design.  Traditional  and  modem  ways  of  handling 
engineering  design  problems  are  compared,  and  key  issues  that  systems 
methodology  and  design  (SMD)  should  address  are  developed.  A  review  of 
several  traditional  sciences  and  disciplines  (e.g.,  psychology,  mathematics, 
philosophy)  shows  that  SMD  can  be  enhanced  and  extended  to  help 
improve  the  results  obtained.  A  merger  of  different  disciplines  and 
pragmatic  action-oriented  perspectives  of  real-world  concerns  leads  to  the 
benefits  of  SMD:  increase  the  probability  of  working  on  the  right  problem, 
provide  a  holistic  view,  identify  the  iterative  process  of  design,  treat  each 
problem  and  recommendation  as  unique,  improve  significantly  the 
likelihood  of  implementation,  and  define  a  solution  as  a  flexible  continually 
changing  set  of  system  specifications.  Recommendations  for  increasing 
engineering  success  with  SMD  include  changing  design  education  for 
undergraduates  to  an  SMD  basis,  relating  management  and  engineering 
concepts,  supporting  increased  funding  for  research,  and  getting  the 
engineering  societies  to  promote  SMD  while  maintaining  an  active  scholarly 
and  research  perspective  so  that  SMD  itself  continually  improves. 

Ostrofsky,  Benjamin,  Morphology  of  Design  of  Aerospace  Systems  with  Inclusion  of 
Human  Factors,  Final  Report  under  AFOSR  Grant  No.  77-3148,  University  of  Houston, 
August  1977,  AD  A049999. 

Pugh,  Stuart,  "Design  Activity  Models:  Worldwide  Emergence  and  Convergence,"  Design 
Studies,  Vol.  7,  No.  3,  July  1986,  pp.  167-173. 


A-34 


VDI  222U  Systematic  Approach  to  the  Design  of  Technical  Systems  and  Products  Design 
Handbook,  Society  for  Product  Development,  Design  and  Marketing,  Committee  for 
Systematic  Design,  translation  of  German  Edition,  November  1986. 

2 .  Miscellaneous 

Bots,  P.W.G.  and  H.G.  Sol,  "An  Environment  to  Support  Problem  Solving,"  Decision 
Support  Systems,  Vol.  3,  No.  3,  September  1987,  pp.  225-231 

The  design  process  as  a  decision  structure. 

Chandrasekaran,  B.,  "Towards  a  Taxonomy  of  Problem  Solving  Types,"  AI  Magazine, 
Vol.  4,  No.  1,  Winter  1983,  pp.  9-17. 

Ericsson,  K.A.,  Simon,  Herbert  A.,  Protocol  Analysis,  Verbal  Reports  As  Data,  The  MIT 
Press,  Massachusetts,  1984. 

Kranzberg,  M.,  "The  Changing  Ecology  of  Innovation,"  Proceedings  of  the  1987 
International  Congress  on  Planning  and  Design  Theory,  Edited  by  G.  Nadler,  Boston, 
MA,  August  1987,  pp.  41-47. 

ABSTRACT:  "Innovation  is  a  process  involving  the  coupling  of  creativity 
and  vision  with  social  needs,  resource  availability,  economic  constraints  and 
opportunities— and  occurring  in  a  social  cultural  environment  which  is 
affected  by  political  policies,  military  requirements,  and  a  host  of  other 
factors.  All  these  elements  interact  with  one  another,  and  they  represent  the 
social  ecology  of  innovation.  This  dynamic  interplay  among  the  many 
elements  comprising  the  environment  of  innovation  has  changed  throughout 
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Structured  Design  methods  of  Tom  DeMarco,  Edward  Yourdon,  and  Larry 
Constantine.  TTie  assessment  is  a  reflection  of  the  author's  experience  using 
Structured  Analysis  and  Structured  Design  on  a  multi-person  project  during 
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development  of  tools  and  procedures  to  improve  problem  solving  and 
decision  making.  This  report  discusses  the  advances  needed  to  combine 
human  thinking  with  intelligent  machines  to  achieve  a  more  productive 
society.  Areas  of  high  potential  include  research  in  expert  systems,  conflict 
resolution,  agenda  setting,  decision  making  in  an  organizational  setting,  and 
empirical  studies  of  individual  behavior.  The  resources  currently  being 
applied  to  research  in  decision  making  and  problem  solving  are  modest  and 


are  not  commensurate  with  the  opportunities  or  the  human  resources 
available  for  exploiting  them. 
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of  Complex  Systems,"  IEEE  Transactions  on  Engineering  Management,  Vol.  EM-28,  No. 
3,  August  1981,  pp.  71-74. 

ABSTRACT:  "Systems  design  involves  the  determination  of  interde¬ 
pendent  variables.  Thus  the  precedence  ordering  for  the  tasks  of 
determining  these  variables  involves  circuits.  Circuits  require  planning 
decisions  about  how  to  iterate  and  where  to  use  estimates.  Conventional 
planning  techniques,  such  as  critical  path,  do  not  deal  with  these  problems. 
Techniques  are  shown  in  this  paper  which  can  be  used  to  develop  an 
effective  engineering  plan,  showing  where  estimates  are  to  be  used,  how 
design  iterations  and  reviews  are  handled,  and  how  information  flows 
during  the  design  work.  This  information  flow  can  be  used  to  determine  the 
consequences  of  a  change  in  any  variable  on  the  rest  of  the  variables  in  the 
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are  all  handled  through  the  use  of  computer  terminals  and  data  bases. 
However,  these  same  techniques  can  also  be  effectively  used  in  classical 
engineering  environments." 
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